DataSnap | Mat DeLong

SOURCE – DataSnap Photo Album Server & Admin Client

Filed under: DataSnap, Delphi, RAD Studio, XE2 — Tags: Ajax, Album, Client-Server, DataSnap, Delphi, RAD Studio, Web, XE2 — mathewdelong @ 7:06 AM

Intro

With my previous blog post (https://mathewdelong./2013/01/24/datasnap-photo-album-server-admin-client/) I shared with you a sample application written in Delphi XE2 using DataSnap to make a client/server application for sharing albums of photos. With this post I want to show you the code and explain parts of it to you.

First off, this is where you can download a zip file of the source: http:///professional/downloads/delphi/AlbumWebsite-v1.0_src.zip

And again, this is where you can download the sample app itself: http:///professional/downloads/delphi/AlbumWebsite-v1.0.zip

Lastly, here is the SourceForge project, in case you want to browse the code that way: http:///projects/albumwebsite/.

I’m going to assume you’ve read my previous blog posts, and so know about the basics of web clients, authentication management, Invocation Metadata, and remote method invocation with DataSnap. If you need a refresher, feel free to read my older blog posts before continuing with this one.

Server Setup

When setting up the server, I chose to create a new DataSnap REST Application, including server methods class, proxy generation and authentication management. The server methods are actually used to load the page content. This is because we need to do some processing of the content before returning it, and this is one way of doing it. The proxy generation is required so that a Delphi Client version of the server methods is available for the admin client, so that it can remotely invoke the server methods. The authentication manager (actually, I’ve used two of them) is to restrict admin functions (like adding new photos and deleting photos/albums) to an authenticated admin user.

Server container form

Above is a photo of the completed server container form. The components I haven’t yet mentioned are the TCP Transport, used by the admin client for uploading photos using a TCP connection, and the file dispatcher. The file dispatcher allows for sending of files through HTTP to the web client, such as when it requests a specific image resource from the server, or a cs file, JavaScript file, etc.

Authentication & Authorization

There are two different types of users who will be connecting to the server: administrators and regular users who wish to view your photos. For this reason, I’ve created user-role constants which will be assigned in the authentication managers for connecting users:

const
gcAdminRole = 'admin';
gcPublicRole = 'public';

And these are the two authentication methods for the two authentication manager components:

procedure TAWServerContainer.WebClientAuthenticatorUserAuthenticate(Sender: TObject; const Protocol, Context,
                                                                    User, Password: string; var valid: Boolean; UserRoles: TStrings);
begin
  UserRoles.Add(gcPublicRole);
  valid := True;
end;

procedure TAWServerContainer.AdminAuthenticatinatorUserAuthenticate(Sender: TObject; const Protocol, Context, User,
  Password: string; var valid: Boolean; UserRoles: TStrings);
begin
  valid := False;

  if AnsiSameText(fAdminPassword, Password) then
  begin
    UserRoles.Add(gcPublicRole);
    UserRoles.Add(gcAdminRole);
    valid := True;
  end;
end;

You can see here that for regular users we just authenticate them, but for the admin user we first check the password. Also, the admin user is granted the roles of both an admin and a regular user. This will allow him to both modify the album and view it like normal. It is up to the TRoleAuth annotations on the server methods to enforce this.

‘Regular User’ Server Methods

These server methods are used to return web page content to regular users. The type of pages returned are a page of all the available albums, and a page for a specific album, showing all of the album photos. Here are the declarations of each:

    /// <summary>
    ///   Results in the HTTP Response containing HTML of a page showing a list of all albums.
    /// </summary>
    [TRoleAuth('public')]
    procedure Albums;
    /// <summary>
    ///   Results in the HTTP Response containing HTML of a page showing the specified
    ///   album's images.
    /// </summary>
    [TRoleAuth('public')]
    procedure Album(const AlbumName: String);

These procedures use the Invocation Metadata to set the HTML of the response to specific HTML, the content of which is built based on the filesystem of the server. The server has a specific directory set as the “albums” directory, and each directory under that is an album. So the names of the folders are the album names returned on the web page. Here is the implementation:

procedure TAWMethods.Albums;
var
  AlbumNames : TStringList;
  AlbumName: String;
  SB: TStringBuilder;
begin
  AlbumNames := GetSubdirectories(lcAlbumsDir);

  SB := TStringBuilder.Create;
  try
    SB.AppendLine('  <div class="titlediv">' + lcTitleAlbumsList + '</div>');
    SB.AppendLine('  <div class="albumlist">');

    for AlbumName In AlbumNames do
    begin
      SB.AppendLine('    <a href="' + lcAlbumAction + '/' + AlbumName + '">' + AlbumName + '</a><br />');
    end;

    SB.AppendLine('  </div>');

    GetInvocationMetadata().ResponseContent := GetPageText(lcAlbumsAction, ptAlbumList, SB);
  finally
    FreeAndNil(SB);
  end;
end;

This code reads the list of subdirectories from the server’s filesystem, and then builds the body of the HTML to return, where there is a link for each subdirectory under the album folder. The links that get built are calls to the “Album” server method, passing in the subdirectory name as the Album name parameter. The link is relative from the Albums URL, so the full path isn’t needed (going from: http://host/DS/rest/Albums to http://host/DS/rest/Album/SomeName). The GetPageText call just adds the HTML body just created to a pre-existing default HTML page with the required links in the header, such as required JavaScript files. Look at the project files for more information on this.

This is what the implementation looks like for the method that returns the HTML for a specific album:

procedure TAWMethods.Album(const AlbumName: String);
var
  ADirPath: String;
  AThumbsPath: String;
  AImageName: String;
  AImageList: TStringList;
  SB: TStringBuilder;
  ACurrentImgPath, ACurrentImgThumbPath: String;
  AAlbumInfo: TJSONObject;
begin
  ADirPath := lcAlbumsDir + '/' + AlbumName;
  AThumbsPath := ADirPath + '/' + lcThumbsDirName;

  AImageList := GetImageList(ADirPath);
  SB := TStringBuilder.Create;

  AAlbumInfo := GetAlbumInfo(AlbumName);

  try
    SB.AppendLine('  <div class="titlediv">' + AlbumName + '</div>');
    SB.AppendLine('  <div class="backlink hcenter"><img src="' + GetWebPath('images/up.png') + '"/><a href="../' + lcAlbumsAction + '">Back to Album List</a></div><br />');
    SB.AppendLine('  <div class="descriptiondiv">' + GetAlbumDescription(AAlbumInfo) + '</div>');
    SB.AppendLine('  <div class="highslide-gallery hcenter"><br />');

    for AImageName In AImageList do
    begin
      ACurrentImgPath := ADirPath + '/' + AImageName;
      ACurrentImgThumbPath := AThumbsPath + '/' + AImageName;

      SB.AppendLine('    <a href="' + GetWebPath(ACurrentImgPath) + '" class="highslide" onclick="return hs.expand(this, galleryOptions )">');
      SB.AppendLine('      <img src="' + GetWebPath(ACurrentImgThumbPath) + '" alt="Loading Thumbnail..." title="Click to enlarge"/>');
      SB.AppendLine('    </a>');
      SB.AppendLine('    <div class="highslide-caption">' + GetImageDescription(AAlbumInfo, AImageName) +'</div>');
    end;

    SB.AppendLine('  </div>');

    GetInvocationMetadata().ResponseContent := GetPageText(lcAlbumAction, ptAlbum, SB);
  finally
    FreeAndNil(SB);
    FreeAndNil(AImageList);
    FreeAndNil(AAlbumInfo);
  end;
end;

The above code works in a similar way to the previous code for loading the list of albums: It scans the file system (this time under the albums subdirectory with the album name) and finds all of the images and thumbnails there. It then builds the body HTML for the album page based on the images it finds. One additional thing it does is looks for a “settings.json” file in the album directory, which holds the description for the album and each image.

The highslide JS library is used in the HTML for loading a full-sized image from a thumbnail, which allows for only loading the full sized images as a user clicks them, which speeds up page loading.

As with before, the last thing this procedure does is use the Invocation Metadata to set the content of the HTTP response. If these methods weren’t called through an HTTP request, then nothing will happen, because the InvocationMetadata isn’t used for TCP connections. These methods are intended to be called from a web browser, with a GET request.

Admin Server Methods

These are the public methods which can be remotely invoked by an authenticated Admin user:

    /// <summary>Returns a JSON Array containing the names of all Albums.</summary>
    [TRoleAuth('admin')]
    function GetAlbumList: TJSONArray;
    /// <summary>Returns a JSONObject for the album, containing image names and descriptions.</summary>
    /// <remarks>Looks like: {"description":"","images":["IMG001.jpg":""]}</remarks>
    [TRoleAuth('admin')]
    function GetAlbumInfo(const AlbumName: String): TJSONObject;
    /// <summary>Returns the image thumbnail, or nil and False if it can't be loaded.</summary>
    [TRoleAuth('admin')]
    function GetThumbnail(const AlbumName, ImageName: String; out ImgStream: TStream): Boolean;
    /// <summary></summary>Renames the specified album.
    [TRoleAuth('admin')]
    function RenameAlbum(const AlbumName, NewName: String): Boolean;
    /// <summary>Sets the info for the given album.</summary>
    [TRoleAuth('admin')]
    function SetAlbumInfo(const AlbumName: String; const AlbumInfoObj: TJSONObject): Boolean;
    /// <summary>Adds image to the specified album.</summary>
    [TRoleAuth('admin')]
    function AddImage(const AlbumName, ImageName: String; FileStream: TStream): Boolean;
    /// <summary>Removes the specified image from the album.</summary>
    [TRoleAuth('admin')]
    function RemoveImage(const AlbumName, ImageName: String): Boolean;
    /// <summary>Deletes the specified album.</summary>
    [TRoleAuth('admin')]
    function DeleteAlbum(const AlbumName: String): Boolean;
    /// <summary>Creates an empty directory with the given name in the albums directory.</summary>
    /// <remarks>This results in an empty Album being created.</remarks>
    [TRoleAuth('admin')]
    function CreateAlbum(const AlbumName: String): Boolean;

These allow for getting, adding, removing and editing of albums and images. The TRoleAuth attribute/annotation is limiting each of the functions to be called only by the admin user (established by the authentication manager.) These methods are used by the Delphi client application for managing albums. For the most part, the implementation of these are quite simple. They use the file system on the server in much the same way the previously mentioned procedures did. For more information on the inner workings, feel free to check out the source code.

Delphi Client

The Delphi client connects to the server through a TCP connection, authenticates with an administrative password, and if successfully logged in, is able to view and modify the list of albums and their contents. This client boils down to simply using a generated proxy for remote method invocation, so I won’t go into more detail on it.

Web Client

The use case for this application is that the person hosting the ‘album website’ wants it to be available for those who are given the URL. The website itself simply consists of the landing page (the list of albums with links to them) and the individual pages for each album. The web client is, therefore, quite simple. There is no need for a generated JS proxy, because the URLs themselves call the server methods. All there is for client code (other than the highslide library and its required files) is a “main.js” and “main.css” file. These files are included by the “GetPageText” function in the server methods class, and the file dispatcher on the server handles delivering them to web browsers when the pages load.

To see the content being built for the web client, view that GetPageText function, and the Album and Albums procedures that call it.

Conclusion

I could have gone a lot more in detail, but I think that with the provided source code, this should be enough information to get you started. Let me know in the comments if you have any questions.

Comments (1)

January 24, 2013

DataSnap Photo Album Server & Admin Client

Filed under: DataSnap, Delphi, RAD Studio, XE2 — Tags: Album, Client, DataSnap, Delphi, HTTP, Server, Web — mathewdelong @ 7:00 PM

A family member wanted an easy solution for hosting lists of photos locally on their PC which others could see in their web browser if they knew the URL. Requirements were simply that no photos could be stored in any cloud service, the page should have a list of thumbnails which can be viewed all at once and individually zoomed in on, and it should be easy to use and update.

After considering the steps required to install and configure a proper web server on his machine (remotely, from another continent) including server-side scripting plugins, etc… I decided DataSnap was the right tool for the job… no need to use a sledgehammer on a tiny nail.

I got to work, and after an afternoon programming session this was the result:

The server form in action.

Login panel of client

Album management from client

The server application is a DataSnap server which handles HTTP requests from a web browser, specifically this URL: http://HOST:8141/ds/rest/TAWMethods/Albums. It also allows for TCP connections, which the client application uses for creating and updating albums. Exposed server methods allow for remote invocation of the required administrative functionalities. (Such as adding an image or deleting one.) The server also allows for setting of an administrative password (authentication manager,) so that not just anyone with your IP and a copy of the client application can modify your album website.

The client application allows for viewing existing albums (including each individual image thumbnail) and adding new albums or photos, adding descriptions to existing photos, or removing photos/albums. It connects to the server using a TCP connection and uses a generated proxy for remotely invoking the server methods.

Here is what the web page looks like in the browser… it is pretty basic. When you click a thumbnail it uses the free ‘Highslide’ JavasSript library to pop up a large version of the image:

Main page, listing all the albums

Example album page

Another example album page

This is a pretty basic application, but I think it shows DataSnap’s beautiful simplicity. I built this quickly and to deploy it to my family member’s PC I just gave them the ZIP file and told them to unzip it anywhere they wanted. If you are interested in the source code leave a comment. If I get enough interest, I might throw together another blog post including the source and some more detail.

Download Here

UPDATE:
Softpedia randomly found this application and featured it on their site!: http://www./get/Multimedia/Graphic/Graphic-Others/Album-Website.shtml

Comments (5)

September 15, 2011

DelphiLive 2011 Recap

Filed under: DataSnap, Delphi, RAD Studio, XE2 — mathewdelong @ 6:13 PM

I have a bit of time now, so I thought I’d write a recap of the two presentations I did this week for DelphiLive. I recorded the presentations, so at some point I may even make these videos available.

My first presentation was on Tuesday, and I covered the new features and improvements in DataSnap XE2. Things like TCP/IP Connection monitoring, Heavyweight callback monitoring (client & server) and various changes to the REST protocol to better accommodate REST clients other than the default DataSnap ones.

My second presentation, yesterday, was about DataSnap Mobile Connectors. These are the new proxy generators for mobile devices. With XE2, you can have a DataSnap server running and generate proxies which will be in the various mobile programming languages. For example, for Android devices Java code is generated, which then allows Android developers to use this as part of their application, for easily communicating with the DataSnap server.

The turnout for this year’s DelphiLive event seemed to be a bit lower than the year before. However, both of my talks had a good turnout, which I was really happy about.

In my previous blog posts I’ve pretty much covered the mobile connectors feature, so with the remainder of this one I am going to be giving a recap of my first presentation, which was the new features and improvements to DataSnap.

Wizard Changes

The first thing I’ll talk about is the DataSnap Server wizard changes introduced in the new release of RAD Studio, and in some cases new features those changes are exposing.

JavaScript Files option on DataSnap Server wizard

The ‘DataSnap Server’ wizard now has a “JavaScript Files” option, which when selected will add the REST JavaScript client files to your project, along with components for generating the proxy, and for dispatching the web files to a web browser running a DataSnap JavaScript rest client. This allows you to easily develop JavaScript REST clients using Indy components, just like you can with the REST Application Wizard. You could do this previously in XE, but you needed to manually add the JavaScript files and components to your project.

A Proxy Generator component will be added, and it will be set by default to automatically generate the JavaScript proxy when your application starts up.

Post-Build Events

Static files, such as the JavaScript framework files, are put into the project’s root directory. Previously (in XE) for the Executable to find these static files, the wizard would generate the project, setting its output directory to be “.”, which would keep it at the same level as the static files.

With XE2, now the output directory for the project is left as .\$(Platform)\$(Config), and a post-build event is added to the project, which will automatically copy the static files into the output directory from the project’s root directory. This post-build event is invoking a batch file which can be found under the RAD Studio install directory, under ObjRepos/en/dsrest/dsPostBuild.bat. You can modify it to change what is copied by the project. You can also change individual projects to invoke a different batch file, or not invoke anything.

Project Location Wizard Page

There is now a Project Location wizard page added to all DataSnap server wizards whenever the project will include static files (either JavaScript files or Mobile Connector files.) This is because the wizard needs to know where to save these files. The project’s units and project file will not be saved automatically, but when you choose to save them, the default save location will be set to the location specified in the wizard. Note that the REST Application Wizard always includes JavaScript files, so will always have the Project Location wizard visible.

Server Module Option

The Web Broker DataSnap server wizards (REST Application Wizard and DataSnap WebBroker Application) now have a “Server Module” option, which will put the TDSServer, TDSServerClass and TDSAuthenticationManager components onto a DataModule unit instead of the regular WebModule one with the other components.

You would want to enable this feature if you wanted to use heavyweight callbacks with the server. Heavyweight callbacks require all users to connect to the same server, but if the server components are on a Web Module, then there will be multiple instances, and each user will be connecting and interacting with their own specific server.

Note that if you want to use heavyweight callbacks with TCP connections, you should instead use the DataSnap Server (Indy) wizard.

HTTPS Protocol

Now with DataSnap standalone servers, HTTPS is an option as a supported protocol. When selected in the wizard, you will just need to specify a port and the appropriate certificate file(s) to use as the server’s HTTPS certificate. HTTPS is enabled by taking the regular HTTP service component and adding specifying a CertFile component in its published property. This CertFile is a component referencing the certificate file(s) on disk. If the Certfile is specified, the HTTP service component uses HTTPS, otherwise it uses regular HTTP.

On the client side, connecting using HTTPS is done the same way as in XE, but now the server no longer needs to be hosted in IIS.

Getting Connecting Client’s Information

On the TDSServer component, you can specify an OnConnect event. There is a new record type called ‘TDBXClientInfo’ which you can get from the ‘TDBXChannelInfo’ stored in the ‘TDSConnectEventObject’ of the OnConnect event. This record contains the IP Address, Protocol and (if possible) application name. Application name is only populated with http protocol, as TCP connections don’t provide this metadata.

Session Events

Adding session events and iterating over available sessions was available in XE, but only worked for HTTP connections. Now this feature is enabled also for TCP/IP connections, and allows you to keep track of all active sessions, and when they are created/closed.

To add a session event, get the singleton instance of the TDSSessionManager class by calling TDSSessionManager.Instance, and then use the AddSessionEvent method to register a procedure with the signature following signature:

TDSSessionManager.Instance.AddSessionEvent(
  procedure(Sender: TObject; const EventType: TDSSessionEventType;
                    const Session: TDSSession)
  begin
    case EventType of
      SessionCreate: (* session was created *);
      SessionClose: (* session was closed *);
    end;
  end);

The procedure takes an EventType parameter, which says if the session was created or is being closed. It also takes the Session itself, to use however you need. (such as storing data in the session, or getting the Session ID [SessionName].)

If you want to iterate over all sessions, you again do that with the singleton instance of the TDSSessionManager class. Using the ForEachSession method, you are able to iterate all of the registered sessions in a thread-safe manner.

TDSSessionManager.Instance.ForEachSession(
  procedure(const Session: TDSSession)
  begin
    //handle Session instance
  end);

Storing TObject instances in a Session

With XE2, sessions can now also store string-TObject pairs by using the Object functions (HasObject, GetObject, PutObject and RemoveObject). This can be used for storing complex data in a session, mapping it with a user’s connection. Note, however, that any object stored in the session is instance-owned by the session and will be freed by the session when it is destroyed.

Heavyweight Callback Monitoring

Server Side

There is a new TDSCallbackTunnelManager class in the DSServer unit. It can be used to keep track of the creation and destruction of heavyweight callback channels, and the callbacks they contain. This allows the server to respond to any state change with the connect clients and their heavyweight callbacks.

You can add and remove events with the following functions:

TDSCallbackTunnelManager.Instance.AddTunnelEvent(Event: TDSCallbackTunnelEvent)
TDSCallbackTunnelManager.Instance.RemoveTunnelEvent(Event: TDSCallbackTunnelEvent)

A specific example:

 TDSCallbackTunnelManager.Instance.AddTunnelEvent(
  procedure(Sender: TObject; const EventItem: TDSCallbackTunnelEventItem)
  begin
    case EventItem.EventType of
      TunnelCreate: (* tunnel was created - EventItem.Tunnel, EventItem.CallbackId *);
      TunnelClose: (* tunnel was closed - EventItem.Tunnel*);
      CallbackAdded: (* callback added - EventItem.Tunnel, EventItem.CallbackId *);
      CallbackRemoved: (* callback removed - EventItem.Tunnel, EventItem.CallbackId *);
    end;
  end);

The tunnel event type is defined as follows:

TDSCallbackTunnelEvent = reference to procedure(Sender: TObject;
                                                const EventItem: TDSCallbackTunnelEventItem);

The TDSCallbackTunnelEventItem type is a record which holds several bits of information, such as the event type (TDSCallbackTunnelEventType) which is from the set:

TDSCallbackTunnelEventType = (TunnelCreate, TunnelClose, CallbackAdded, CallbackRemoved);

The event type also holds the tunnel instance, its ID and server channel name (for convenience, or if the tunnel instance itself is unavailable,) and optionally the callback ID and callback’s channel names.

Client Side – DBX

Similar to the heavyweight callback server events just mentioned, you can also register heavyweight callback events on the client. This doesn’t notify you for all other clients connected to the server, just your own client. Specifically, just the TDSClientCallbackChannelManager (for DBX clients) the event is registered on.

DBX Client events are very similar to the server ones, but have an additional event type possibility: TunnelClosedByServer. If the server initiated the closing of the heavyweight callback, this will be used as the event, instead of TunnelClose.

Also, the EventItem contains a callback item instance, which wraps the callback and can be sued to get the callback’s unique ID and list of server channels being listened on.

Client Side – Delphi REST

Very similar to the DBX client monitoring, but for REST Clients written sing the TDSRestClientChannel class. TDSRestClientChannel has a property on it called “OnChannelStateChange” which can be set to a procedure which takes a single TDSRESTChannelEventItem parameter.

This event contains the event type, which is a different set from the DBX and server event types (TDSRESTChannelEventType.) The types are similarly named, but prefixed with “r”.

Client Side – JavaScript REST

JavaScript clients also support heavyweight callback events. The ClientChannel JavaScript call now has a field called “onChannelStateChange” which can be set to a function handle.

var channel = new ClientChannel (clientID, channelName);
channel.onChannelStateChange = HandleChannelEvent;

function HandleChannelEvent(EventItem) { //ClientChannelEventItem
  switch(EventItem.eventType) {
      case EventItem.channel.EVENT_CHANNEL_START:
        alert("Channel Started: " + EventItem.channel.channelId + ", Callback: " + EventItem.callback.callbackId); break;
      case EventItem.channel.EVENT_CHANNEL_STOP:
        alert("Channel Stopped: " + EventItem.channel.channelId); break;
      case EventItem.channel.EVENT_CALLBACK_ADDED:
        alert("Callback Added: " + EventItem.callback.callbackId); break;
      case EventItem.channel.EVENT_CALLBACK_REMOVED:
        alert("Callback Removed: " + EventItem.callback.callbackId); break;
      case EventItem.channel.EVENT_SERVER_DISCONNECT:
        alert("Channel disconnected by server: " + EventItem.channel.channelId); break;
    }
}

This function will be invoked whenever the channel is stated or stopped, and whenever a callback is added to or removed from the channel.

The function being set for the onChannelstateChange event should take a single parameter, which will be an instance of ClientChannelEventItem. This class contains an “eventType” field, as well as a “channel” and “callback” field.

TCP Connection Monitoring

With RAD Studio XE2, DataSnap servers with TDSTCPServerTransport components are able to monitor connections, and close any TCP connection they wish. The connections are linked with a Session Id, which can be used in a server method or authentication manager to get the TCP connection for the current session. This allows both server methods and authentication managers to terminate a connection for any reason.

On the TDSTCPServerTransport component there are two new events which can be assigned; OnConnect and OnDisconnect.

An implementation of the OnConnect event may look like this:

procedure TForm1. ServerTransportConnectEvent(Event: TDSTCPConnectEventObject);
begin
  //Add both the connection and Channel (TDSTCPChannel) to a dictionary for later use
  FConnections.Add(TIdTCPConnection(Event.Connection), Event.Channel);
end;
 

The above code captures and stores the Channel, which is an instance of TDSTCPChannel, as well as the connection for later use. The connection will be provided without a channel in the disconnect event, so in a way it can be used to uniquely identify a channel. The channel has a GetConnection function which will return its connection if needed.

An implementation of the OnDisconnect event may look like this:

procedure TForm1. Server TransportDisconnectEvent(Event: TDSTCPDisconnectEventObject);
begin
  //Remove the connection and its associated channel from the dictionary
  FConnections.Remove(TIdTCPConnection(Event.Connection));
end;

The above code is called whenever the TDSTCPServerTransport component is still active and a TCP connection has been closed. If the transport is being disposed, then this event will not be notified. Note that the event provides only the connection, but this connection could be used to look up a channel you’ve obtained with the OnConnect event.

By default, the OnDisconnect event will not be notified if the client abruptly loses his internet connection. This is because the socket remains open until an IO operation is attempted and fails. If your OS is configured to use keep-alive packets for all TCP/IP connections then based on its configuration, you will eventually see the disconnect event being notified. If you’d like to control this behavior on a per-connection basis, then you can use the EnableKeepAlive and DisableKeepAlive methods on the TDSTCPChannel:

//If the connection is idle for 10 seconds, then send a keep-alive packet to check if the client is still there.
Event.Channel.EnableKeepAlive(10000);

The above code will enable keep-alive for the specific channel/connection. This will send a keep-alive packet to the client whenever it has been idle for more than the specified time (10 seconds.) If the client does not respond then the packet will be resent a number of times, as defined by the Operating System. (For example, in Windows 7, it will retry 10 times.)

There is an optional second parameter to the EnableKeepAlive procedure, which is an integer representing the number of milliseconds to wait between retries if a client doesn’t respond to a packet. If undefined, it defaults to a value of 100 milliseconds.

If you wish to disable keep-alive for a specific connection, then get the Channel instance and call DisableKeepAlive.

You can close a connection at any time by getting the connection’s channel, and calling its Close procedure. An example could look like this:

//Get the associated Channel for the given connection, and if successful close it
if FConnections.TryGetValue(Connection, Channel) then
  Channel.Close;

JavaScript Changes

With XE2 json-min.js has been replaced with json2.js. This new implementation uses static functions which take in variables to parse, instead of adding prototypes to all JavaScript objects, which don’t play nice with many jQuery plugins.

Now to parse a string into its JavaScript equivalent, you use: JSON.parse(“json_string”)

Similarly, if you have a JavaScript object you want to put into JSON format, you use: JSON.stringify(jsonValue)

Another change in the JavaScript code is that the initSessionData function (which allows the SessionID to be stored in a cookie and remembered between pages/loads of a web app,) takes an optional second parameter called “sessionCookiePrefix”. This new parameter prefixes the name of the cookie key used by the application. This allows you to have multiple web apps running at the same time, without overwriting the session ID cookie of eachother. The apps just need different cookie prefixes to distinguish the cookies.

Also, and not just restricted a JavaScript change, the Broadcast function of ClientChannel takes ChannelName as an optional seciond parameter. This is because individual callbacks within a ClientChannel/Tunnel now can specify one or more server channel names they listen on. The ClientChannel‘s ChannelName is now optional, and you may want to broadcast to a different channel other than that one.

The ClientCallback class now takes an optional serverChannelNames parameter, which is a comma seperated list of server channel names that specific callback listens on. If this is left as an empty string ro null, then the callback just listens on whichever channel its parent ClientChannel does.

FormatResult Event for REST Responses

The TDSHTTPService and TDSHTTPWebDispatcher components have a FormatResult event, which allows for modifying the JSON Result being passed back to a client. You have access to the command being invoked by the client, and the JSON response (minus the result JSON Object) the client will be getting.

The ResultVal in most cases will be a JSON Array. This array will have a value for each out/var/Return parameter of the server method being invoked.

The Command parameter is the command being invoked. Command.Text will give you a string representation of the method being invoked, such as “TServerMethods1.EchoString”

The value of Handled is false by default. If set to true, then the result passed to the user will not be wrapped in a “result” JSON object. If it is false, then it will be wrapped in this object.

This formatResult event allows you to completely control the format of the JSON passed back to the client, which is very important when 3rd party libraries are interacting directly with the server, and expect a very specific response format. The jqGrid jquery plugin, for example.

The following is an example of a FormatResult implementation which returns the result of a server function call directly, without wrapping it in an array or JSON Object:

procedure TServerContainer1.DSHTTPServiceTest1Result(Sender: TObject; 
              var ResultVal: TJSONValue; const Command: TDBXCommand; 
              var Handled: Boolean);
var
  Aux: TJSONValue;
begin
  if Command.Text = 'TServerMethods1.EchoString' then
  begin
    Aux := ResultVal;
    ResultVal := TJSONArray(Aux).Get(0);
    TJSONArray(Aux).Remove(0); //remove the item so it isn't disposed when array is freed
    Aux.Free;
  end;
end;

REST Query Parameter Support

In a server method being called by an HTTP request, you have access to the parameters by calling:

GetInvocationMetadata().QueryParams

Or, for example:

function Class.Method(String p1) {
  //p1 is ignored in this example
  Result := GetInvocationMetadata().QueryParams.Values['key1'];
}

Note that you need to add the Data.DBXPlatform unit to your uses clause to have access to the invocation metadata.

The QueryParams property is a TStrings instance which holds key value pairs for all of the query parameters passed in through the URL of the REST call.

Again, this functionality is crucial for interactions with 3rd party libraries, such as jQuery plugins. They may need to send query parameters to the server, and expect these to affect the result.

New Cloud API

The Azure API has been redesigned since the release of XE. In XE2 the old API (and visual components) have been deprecated, and you should instead use the new API, found in the Data.Cloud.AzureAPI unit. This API is designed to be easier to understand and easier to use. It is fully code commented, and should be pretty self explanatory. It still works with the same services as the old Azure API: Azure Queue Service, Azure Table Service and Azure Blob Service.

XE2 also has a new Amazon API, which interacts with Amazon’s AWS services, similar to Azure’s: Amazon Simple Queue Service (SQS), Amazon SimpleDB and Amazon Simple Storage Service (S3.) Again, this API is designed to be easy to understand and use, and is fully code commented.

You can use both of these cloud APIs, along with your login credentials to access your Azure and Amazon accounts. You can create queues, add messages to queues, and pop messages from queues. (for example.) You can work with both cloud services’ NOSQL Database service, to store and retrieve data from tables stored on the cloud. And you can use the blob/storage service cloud APIs to upload files to the clouds and download files from the clouds.

In Summary

I think this is by far my longest blog post yet! I could have probably spread it out over 10 posts, but then it wouldn’t have really been a DelphiLive summary. I hope you’ve found some of this useful!

Comments Off on DelphiLive 2011 Recap

May 30, 2011

Heavyweight Callbacks

Filed under: DataSnap, XE — mathewdelong @ 12:54 PM

In a post from last year (DelphiLive Presentation Summary) I discussed Heavyweight callbacks with JavaScript REST clients. I’d now like to take the opportunity to go over using Heavyweight callbacks with Delphi REST and Delphi native clients.

Notes before starting

• If your server project has been created with the REST Application Wizard (uses a Web Module) you will need to create a DataModule unit and move your TDSServer and TDSServerClass components onto it. This is because a single instance of TDSServer needs to be created for all callbacks to register against, but Web Modules would create a new instance per request.
• If your server uses a WebModule and DataModule configuration, as described above, you can not use heavyweight callbacks with Delphi Native clients. To accomplish this, you should instead use the regular DataSnap server wizard, and add any additional required components and the web files, if needed.
• In RAD studio XE, there is no way to provide Authentication Information with a Delphi native heavyweight callback (TDSClientCallbackChannelManager).

Videos

I’ve recorded these videos on working with heavyweight callbacks, which you may find helpful:
Heavyweight Callbacks with DataSnap – Part 1: Thick Client
Heavyweight Callbacks with DataSnap – Part 2: Thin Client

Delphi REST Clients

Creating a Delphi client application which communicates with a DataSnap server using the REST messaging protocol is a snap (pun intended.) Simply drop a TDSRestConnection onto your form, create a TDSRestClientChannel instance using that connection, and then register a callback.

The one thing to note is that the first callback registered must be set with a call to “Connect” on the channel, while every callback after that is registered with a call to “RegisterCallback”.

The code for creating the channel and registering a callback could look like this:

FManager := TDSRestClientChannel.Create('someUniqueID', 'MemoChannel', RestConn);
...
LCallback := TDSRestClientCallback.Create(Self.FManager, 'AnyName',
      function(AValue: TJSONValue; ADataType: string): Boolean
      begin
        LogMessage(AValue.ToString); //do something with a string representation of the message
        Result := true;
      end
    );
if FManager.Connected then
  FManager.RegisterCallback(LCallback)
else
  FManager.Connect(LCallback);

And that’s it! If your server is set up properly (broadcasting to ‘MemoChannel’), and if the TDSRestConnection component has the host and port configured, then your callback will be registered and listening for calls from the server.

You can broadcast and notify messages by using the corresponding functions on the TDSRestClientChannel instance.

Delphi Native Clients

If you wish, instead of using the REST protocol, you can use a TDSClientCallbackChannelManager component to register heavyweight callbacks. With this component you can choose any of the available protocols: http, https or tcp/ip. If you choose http or https, then the TCP/IP traffic will be wrapped in HTTP requests/responses. You will probably find this a bit slower as there is an overhead to wrapping the traffic, but depending on your network configuration you may find this necessary.

The channel manager component has a few properties which need to be set: First, you should set the ChannelName property, which specifies which channel on the server to listen to. For this example, set it to MemoChannel to be the same as the REST Connection mentioned above (and previous examples of the DataSnap server.) Then set the protocol, host and port to match your server connection information.

Next, you want to register a callback. To do that you first define a class (required DBXJSON unit):

  TMyCallback = class(TDBXCallback)
  public
    function Execute(const Arg: TJSONValue): TJSONValue; overload; override;
  end;

Make sure the implementation of Execute returns a value, such as “TJSONTrue.Create”. And then register the callback:

Success := ClientCallbackChannelManager1.RegisterCallback('AnyName', TMyCallback.Create); //returns true if successful, false otherwise

If no callbacks have yet been registered with the client channel, the new callback instance will be used as the ‘first callback’ and the channel will be established with the server. Otherwise, if the channel is already established, the new callback will be added to it.

The channel manager component has broadcast and notify functions, which allow you to send messages to other clients. These are similar to the ones I’ve previously mentioned for the JavaScript REST heavyweight callbacks.

Comments Off on Heavyweight Callbacks

January 18, 2011

Invocation Metadata

Filed under: DataSnap, XE — mathewdelong @ 5:56 PM

Within a Server Method you have the ability to set invocation metadata. I agree, this could be better named. What it does is provide a way to control the HTTP response code (and/or content) returned for an HTTP request invoking the method.

This can be used, for example, to return a 400 (Bad Request) code (or any other code) if one of the parameters passed to the method fails some type of requirement. This could be an integer not falling within a specific range, or a string not being one of the allowed values… or really any other requirement you can think of. You can also use any data stored in the current session (see my previous post on getting the current session and working with its data,) to determine the status code to set.

Here is a simple example of the Invocation Metadata in use:

function TServerMethods1.EchoString(Value: String): String;
begin
  if AnsiSameText("Hello World", Value) then
    Result := Value
  else
    GetInvocationMetadata().ResponseCode := 400; //Bad Request
end;

NOTE: You must specify DBXPlatform in your uses clause to have access to GetInvocationMetadata().

This will result in the following JSON being returned as the content of the message:

{"result":[""]}

The HTTP response code will be 400 and the HTTP response message will have been automatically set to the default 400 message (Bad Request).

With the Invocation Metadata you can also control the content of the response. For example, you could modify the previous code snippet like this:

function TServerMethods1.EchoString(Value: String): String;
begin
  if AnsiSameText("Hello World", Value) then
    Result := Value
  else
  begin
    GetInvocationMetadata().ResponseCode := 400; //Bad Request
    GetInvocationMetadata().ResponseContent := '{"error":"Message to echo must be ''Hello World''"}';
  end;
end;

This does the same as the previous example, but also changes the default response content returned to be a JSON Object containing an error message, instead of the default ‘result’ JSON Object pair.

Comments Off on Invocation Metadata

November 30, 2010

Server Side Session Management

Filed under: DataSnap — mathewdelong @ 10:29 AM

When a client connects to a DataSnap server, a session is created. This session is represented with a TDSSession instance or subclass.

Session Lifecycle

Setting SessionTimeout

For TCP connections the session ends only when the connection is closed. For HTTP connections how long a session is alive for can be controlled by the SessionTimeout property exposed by either the TDSHTTPServerTransport or TDSHTTPServer class. For example, the TDSHTTPService component publishes this property, as does TDSHTTPWebDispatcher. The value is set in milliseconds, and represents the amount of time which is allowed to pass of inactivity for a session before the session expires. Whenever a client issues a request to the server providing his session id, the session is marked as active at that time, and the clock is reset for when the session will expire.

Closing a Session

To close a session, you need to know the SessionId (TDSSession.SessionName, actually). To close the session, simply call:

TDSSessionManager.Instance.CloseSession(SessionId);

Getting the Current Thread’s Session

From a server method, for example, you can obtain the current thread’s session. This may provide you with useful information relating to the user issuing the current request. To do so, use the following code:

Session := TDSSessionManager.GetThreadSession;

Listening for Session Creation and Session Expiry

You can register an event with the TDSSessionManager, which will be notified when new sessions are created and old ones expire. You do so with the following code:

TDSSessionManager.Instance.AddSessionEvent(
  procedure(Sender: TObject;
            const EventType: TDSSessionEventType;
            const Session: TDSSession)
  begin
    case EventType of
      SessionCreate:
        {The provided Session was just created.}
      SessionClose:
        {The provided Session has just been closed, either intentionally or it has expired.}
    end;
  end);

If you want to later remove the event, store it in a field, and later call RemoveSessionEvent passing the event as the parameter.

NOTE: In XE this feature is currently only available with HTTP connections. In XE2, it also works for TCP/IP.

Session Data

You can store data within a session. The data is stored in key/value pairs, where both the key and the value are strings.

Storing Data

To store data in a session, call PutData, passing in the key and the value to store.

Retrieving Data

To see if a value for a specific key exists, call HasData. To get a value, call GetData passing in the key as the parameter.

Removing Data

To clear a stored key/value pair from the session data, call RemoveData passing in the key as the parameter.