Effective ClientDataSets and the BriefCase Model
In this paper, we cover the ClientDataSet component in three situations: using the stand-alone MyBase format; using dbExpress; and finally, using a client-side "briefcase" for DataSnap
multi-tier applications.
I'll use Delphi 7 as well as Delphi 8 (for .NET) and Delphi 2005 to
illustrate the use of ClientDataSet in VCL (for .NET) applications.
However, similar techniques can be applied with Kylix (on Linux) and
C++Builder.
1. Stand-alone ClientDataSet
The stand-alone ClientDataSet is the perfect solution for situations
where standalone executables are required without the need to install
additional database engines or drivers.
Since a ClientDataSet loads all the data in memory it's very fast, but
the size of the database tables is limited to the amount of available
memory (and the load/save times also increase correspondingly).
Furthermore, we cannot perform SQL queries, but filters and other
operations are fine.
Why Local ClientDataSets?
The benefits of using ClientDataSet as local DataSet over, for example,
the BDE are numerous.
First of all, the ClientDataSet is contained within a single DLL called
MIDAS.DLL.
Just drop it inside the Windows\System directory (or leave it in the
same directory as your Delphi executable) and you're in business.
With Delphi 6, you can even add the MidasLib unit to your uses clause
which embeds the entire MIDAS.DLL inside your executable (which will
grow by about 200K), resulting in a true stand-alone zero-configuration
executable! Compare this to installing the BDE on your client machine.
And even it you decide to use ADO, which will probably be on your
client's machine already, you need to make sure the actual database
backend (Access, SQL Server) is also present and accounted for.
In short, the MIDAS.DLL is probably one of the easiest to install
database engines I've seen so far.
And the ClientDataSet component, implemented by this MIDAS.DLL is also
one of the fastest DataSet implementation I've seen.
Sorting, filtering, all with blazing speed (we'll see some examples of
this next time, when we actually start using local ClientDataSet
components in applications).
How come it's so fast? Well, that's one of the potential downsides of
the ClientDataSet.
You see, everything is managed in memory.
And every operation such as sorting, filtering or searching is also done
in memory.
Which explains the speed.
But which also means that for a really large (amount of data in your)
ClientDataSet, you also need a large amount of memory in your machine.
Feeding Local ClientDataSets
The easiest way to load a ClientDataSet at design-time is to right-click
on the ClientDataSet component and select the "Assign Local Data"
pop-up menu option.
This will show a dialog that lists all available DataSets (Tables,
Queries, etc) that are available on the form or data module itself.
If you pick one, then all data from that DataSet will be assigned to the
ClientDataSet.
You can then remove the actual (source) DataSet from the form or data
module, and be left with a stand-alone ClientDataSet only.
Note that you still need to remove the DBTables unit if you used a BDE
Table or Query (and want to make your new application BDE-less).
Apart from using the "Assign Local Data" option, a ClientDataSet can
also load and store its information on disk.
This is visible at design-time by the "Load From File" and "Save To
File" pop-up menu options.
The ClientDataSet component itself also contains these methods, as well
as LoadFromStream and SaveToStream methods, which you can redirect to
different kinds of streams (like a TMemoryStream, right before sending
this stream over a socket connection, for example, but we'll look at
that in more detail next time).
ClientDataSet can load and store two kinds of data formats.
The first one is typically called "cds" format, and is the internal (and
undocumented) binary format.
Small, native and almost impossible to share (except with other
ClientDataSet components from Delphi 5 and higher, C++Builder 5 and
higher, and Kylix).
The second data format that ClientDataSets support is XML.
And we've all heard that XML means eXtended Markup Language, and is
often used in the same sentence as "open", "cross-platform" and
"portable".
However, the XML format that is used by the ClientDataSet is still a
propriety format defined by Borland, so it's not easy to let it be used
by an ADO dataset, for example.
Using Local ClientDataSets
Once a local ClientDataSet is filled with data, we can navigate through
it just like any other dataset.
However, there is a difference, which can be both a benefit and a curse
(if you're not aware of it, that is).
The main difference between the TClientDataSet and the (BDE) TTable is
the fact that the TClientDataSet doesn't automatically save its contents
to disk.
And even if it does, it only saves the changes, and doesn't really apply
them to the dataset.
What does that mean, exactly, and how can we make best use of this
functionality? First of all, let's take a look at the saving
capabilities of the TClientDataSet.
If the data inside the TClientDataSet is loaded by any other means than
the FileName property, then obviously the TClientDataSet doesn't know
where to save the data if it's modified (the original data is streamed
in from the .DFM file).
If the FileName property is used, then the TClientDataSet will save its
contents back to the file when it's explicitly deactivated (or
destroyed).
However, in some situations it may be a good idea to explicitly call the
SaveToFile method (just as you would then have to use LoadFromFile to
load the new data back in again).
Now, if you use LoadFromFile at the start of your application, and
SaveToFile at the end of it, then you may notice that the external file
(with the TClientDataSet contents) grows.
At a higher rate than you would expect from the few changes and possible
appends that you make in the TClientDataSet.
Something is going on inside this file, and if you've saved it in the
XML format, then you can quickly find out what: all individual changes
are saved, with both the "original" value of the record and the
"changed" value of the record.
This means that even for a few changes, the database quickly contains
multiple versions of records with changes, which take up more room than
the changes themselves (or the changes applied to the dataset).
Why is this done? Basically, to enable the TClientDataSet to work in a
multi-tier (and multi-user) environment that we'll examine later.
In a multi-tier environment, the TClientDataSet needs to call the
ApplyUpdates method, sending pending updates to the remote dataset tier.
But this needs to include both the original version of the record(s) as
well as the changes that are to be applied.
If the original version of the record(s) doesn't match the actual
version of the record(s), then the update may not be applied.
This is a typical multi-user problem that had to be solved, to avoid
database integrity problems.
But when using TClientDataSet as a lean and mean stand-alone solution,
it's all of a sudden not so lean and mean anymore.
It gets worse once you realise that the TClientDataSet contains the
original value of all records as well as all changes, which means that
when you load the external file, it has to re-apply all changes (which
can take time as well, so it will load slower and slower over time, as
more and more changes are applied).
MergeChangeLog
Fortunately, there is a special method called MergeChangeLog which does
just as the name implies: it merges all changes in the TClientDataSet,
resulting in a very small file again (with only the records and all
changes applied to them).
Obviously, this method should never be called in a multi-tier
environment, since it would break all possible calls to ApplyUpdates
that you want to make.
But in a single-tier local environment, this method is just fine, as it
shrinks down (and speeds up) the local representation of the
TClientDataSet.
Before calling MergeChangeLog, it wouldn't hurt to check on the value of
ChangeCount, which represents the number of changes that are currently
available in the TClientDataSet.
Only if ChangeCount is greater than zero, you should call MergeChangeLog
(otherwise it's just a waste of time).
As last tip, you may want to consider the fact that a change log does
include the ability to "undo" changes on a record by record basis using
the UndoLastChange, CancelUpdates or RevertRecord methods.
See the on-line help for more details.
These methods can also be used in a multi-tier environment, of course
(before you call ApplyUpdates).
2. ClientDataSet and dbExpress
When combined with dbExpress, the ClientDataSet component can be seen as
the 'cache' (or briefcase) in which the unidirectional cursors can put
their data, so users can view them in a multi-directional buffered
(cached) set of records.
This enables SQL capabilities, since the dbExpress layer (instead of the
ClientDataSet) is now providing the data.
We should now no longer rely on local storage, but call ApplyUpdates to
send the changes to the actual database.
What is dbExpress?
dbExpress is a cross-platform, light-weight, fast and open database
access architecture.
A dbExpress driver must implement a number of interfaces to get
metadata, execute SQL queries or a stored procedure, and returning a
unidirectional cursor.
We get back to this in a moment.
The Kylix Component Library is called CLX (Component Library for
X-platform).
And CLX is divided in four parts: BaseCLX, VisualCLX, NetCLX and
DataCLX.
A question that comes up often is where exactly dbExpress fits in.
Obviously, dbExpress and DataCLX are connected.
And in fact, that's exactly what's happening: dbExpress is the low-level
database access driver, where DataCLX is a set of components that
connect to these drivers.
Not the visual data-aware components, mind you, these are part of
VisualCLX, but the data access components that we can use to
specifically work with the data (regardless of the differences in the
underlying DBMS or flat file).
Custom dbExpress
And that's not all, because dbExpress was created as an Open Database
Architecture, meaning that anyone can write a dbExpress compatible (or
is that called compliant?) driver for use with Kylix and Delphi 6-7 or
C++Builder 6.
An article about the dbExpress internals by Ramesh Theivendran, the
architect of dbExpress, was published on the Borland Community website.
Although this was just a draft specification, it made it clear that
anyone can write one.
As a practical example, Easysoft has developed a dbExpress Gateway for
ODBC, which can be used to connect to UNIX ODBC, and - via their
ODBC-ODBC Bridge - even to a remote Microsoft SQL Server, Access or
other Windows ODBC driver.
Components
The dbExpress tab contains seven components: TSQLConnection,
TSQLDataSet, TSQLQuery, TSQLStoredProcedure, TSQLTable, TSQLMonitor and
TSQLClientDataSet.
TSQLConnection
The TSQLConnection component is literally the connection between the
dbExpress drivers and the other DataCLX components.
If you drop this component on a form or data module, you see only 12
properties.
The one that's probably most used is the ConnectionName property, which
can be assigned with one of the values from the drop-down combobox.
On my Kylix installation, I have the choice of DB2Connection, IBLocal,
MySQLConnection and OracleConnection.
If you select the IBLocal value, then the DriverName property gets the
value INTERBASE, the GetDriverFunc property gets the value
getSQLDriverINTERBASE, the LibraryName property gets the value
libsqlib.so.1 and the Vendorlib property gets the value libgds.so.0.
All automatically, based on the value IBLocal for the ConnectionName.
You can open the Params string list editor to edit the values of the
parameters.
These are also automatically filled in, by the way, when you select a
value for the ConnectionName property.
If you do not want this to happen, for example when writing some
non-visual code to access databases and you want to provide your own
parameter values, then you can set the LoadParamsOnConnection to False.
If you right-click on the TSQLConnection component, you'll see the Connection Settings for the four different Connection Names.
Once you have everything set right, you can set the Connected property
to True (and either get an error message when the database cannot be
found, or see the property get the value True indeed for success).
TSQLDataSet
Once you have a connected TSQLConnection component, you can use any of
the other DataCLX components, such as the TSQLDataSet, which is the most
"general" of these components.
Always start by setting the SQLConnection property of this component to
(one of) the available TSQLConnection component(s).
The TSQLQuery, TSQLStoredProc and TSQLTable components can be seen as
special instantions of the TSQLDataSet component.
In fact, this reminds me a lot of ADOExpress, in which the TADODataSet
component is the "mother" of the TADOQuery, TADODataSet and TADOTable
components.
And both the Delphi ADOExpress and dbGo for ADO, and the Delphi &
Kylix dbExpress TxxxDataSet "core" components share the CommandType and
CommandText properties, with which you can determine the sub-type of the
component.
If you set the value of the CommandType property to ctQuery, then the
CommandText property is interpreted as SQL query.
If you set the CommandType to ctStoredProc, then the CommandText
specifies the name of the stored procedure.
And finally, if you set CommandType to ctTable, then CommandText
contains the name of the individual tables.
In our case, using the general TSQLDataSet component, we can set the
CommandType to ctTable, and the CommandText to customer to select the
customer table.
If you set the Active property to True, you get live data at
design-time, just as we've been used to with Delphi (and if you set the
LoginPrompt property of the SQLConnection component to False, you don't
even see the login dialog).
Nothing special, nothing different.
Yet.
Unidirectional!?
We can now move to the Data Controls tab of the component palette, and
start using some of these to display the data we receive from the active
TSQLDataSet component.
Note that we cannot use all of these components right now without some
special considerations.
This is the place where the biggest difference between the BDE and the
dbExpress architecture is present.
TSQLDataSet (and the derived components TSQLQuery, TSQLStoredProc and
TSQLTable) returns a unidirectional cursor.
Meaning that you can move forward, but not backward.
Which isn't useful when using in a TDBGrid (we can only see one record
at a time!), and watch out when using a TDBNavigator as well, because
clicking on the Back or First button will raise an exception!
Why a Unidirectional cursor? Well, the obvious answer is speed.
The Borland Database Engine has never been our best friend (let's call
it a good friend, or a friendly relative), but it has helped us with the
small and simple database needs.
Unfortunately, the BDE footprint and overhead hasn't been small.
And BDE tables have never been known for their amazing speed.
And that's an area where Borland wanted to show some real improvements.
The new architecture called dbExpress is designed with this in mind.
And hence unidirectional cursors as resultset, with no overhead for
buffering data or managing metadata.
A unidirectional cursor is especially useful when you really only need
to see the results once, or need to walk through your resultset from
start to finish (again once), for example in a while not eof loop,
processing the results of a query or stored procedure.
Real-world situations where this is useful include reporting and web
server applications that produce dynamic web pages as output.
But especially when combined with visual data-aware controls, you
quickly realise that in a GUI driven environment, the user will want to
go back one record.
So you need to somehow cache these records in order to be able to show
them in a grid and to browse back as well as forward.
That's where the TClientDataSet comes in.
It's entirely possible and sensible to use a TDataSetProvider (from the
Data Access tab of the Component Palette) to hook up with the
TSQLDataSet component, and then use a TClientDataSet to obtain its
records from this TDataSetProvider.
The result is a ClientDataSet that gets its records (once) from a
unidirectional source: the SQLDataSet.
The DataSetProvider is only use as a local transportation means.
TClientDataSet
The fact that the TClientDataSet component is available in Delphi as
well as Kylix means a quick and easy way to migrate local database
tables (such as, indeed, BDE tables in Paradox or dBASE format).
This is the first way in which you can migrate from the BDE to
dbExpress: migrating data.
The second way is by migrating the application as well.
To continue now with the way to migrate data from the BDE to a native
ClientDataSet format, consider the code below for a new utility called
dbAlias that I've written, which will convert all tables from a given
(command-line passed) alias to XML files:
{$APPTYPE CONSOLE}
program dbAlias;
uses
Classes, SysUtils, DB, DBTables, Provider, DBClient;
var
i: Integer;
TableNames: TStringList;
Table: TTable;
DataSetProvider: TDataSetProvider;
ClientDataSet: TClientDataSet;
begin
TableNames := TStringList.Create;
with TSession.Create(nil) do
try
AutoSessionName := True;
GetTableNames(ParamStr(1), '', True, False, TableNames);
finally
Free
end {TSession};
Table := TTable.Create(nil);
DataSetProvider := TDataSetProvider.Create(nil);
ClientDataSet := TClientDataSet.Create(nil);
try
Table.DatabaseName := ParamStr(1);
for i:=0 to Pred(TableNames.Count) do
begin
writeln(Table.TableName);
Table.TableName := TableNames[i];
Table.Open;
DataSetProvider.DataSet := Table;
ClientDataSet.SetProvider(DataSetProvider);
ClientDataSet.Open;
ClientDataSet.SaveToFile(ChangeFileExt(Table.TableName,'.xml'));
ClientDataSet.Close;
Table.Close
end
finally
Table.Free
end
end.
This is a quick-and-dirty way to convert your existing BDE aliases
containing Paradox and dBASE files to XML, after which you can put these
files on a Linux box (using FTP, a network connection or even a floppy
disk) and load them in Kylix using a TClientDataSet component.
Obviously, this code only compiles in Delphi, and not in Kylix.
dbExpress Updates
Delphi 6-7 and Kylix both use the new cross-platform dbExpress data access layer.
But how do we ensure that data in our datasets is updated correctly? (and we don't miss any updates or other changes).
I will now show that using dbExpress we must adopt a new way of looking
at data (and especially at saving data), because dbExpress provides
read-only unidirectional datasets only (so we no longer have automatic
posts or updates to our local tables).
Read-Only Uni-Directional
To illustrate this point (and show how we should proceed), let's use Delphi 6 or 7 to build a dbExpress application.
First, do a File | New and select CLX Application (a
cross-platform application, which will also compile on Linux using
Kylix).
Drop a TSQLConnection component on the CLX form, and set its
ConnectionName to IBLocal.
You may have to right-click on the SQLConnection component to start the
Connections Editor in order to make sure the database is pointing to an
actual physical InterBase gdb file.
And of course the password for SYSDBA is masterkey (just in case there's
someone in the community who doesn't know that, yet).
Next, drop a TSQLTable component and set its SQLConnection property to
the SQLConnection1 component.
Select one of the available tables in the TableName property.
We now have a read-only unidirectional dataset (which supports moving
one step ahead or all the way back to the beginning, but no other
operations).
This is nice when connecting to a DataSetTableProducer component (where
we only need to walk through the resultset just once anyway), but not so
useful in most other situations.
ClientDataSet
In order to display the information from the TSQLTable (or any dbExpress
dataset), we need to cache it inside a TClientDataSet component, using a
TDataSetProvider component as "connector".
So, drop both a TDataSetProvider and a TClientDataSet component from the
Data Access tab of the Component Palette.
Assign the SQLTable component to the DataSet property of the
DataSetProvider, and then assign the name of the DataSetProvider to the
ProviderName property of the ClientDataSet.
As soon as you open the ClientDataSet (for example by setting the Active
property to True), the content of the TSQLTable will be traversed (just
once) and the records in the resultset will be provided to the
ClientDataSet, which will cache them from that moment on.
We can now use a DataSource and (for example) a DBGrid component to
display the contents - provided by the ClientDataSet component.
Update
The update problem occurs when we run the resulting application, make
some changes to some of the fields and records, and exit the application
again.
When we restart, we see the old values again!
The ClientDataSet is great for caching, but did not update the actual
database for us (automatically).
Since the TSQLTable component itself is read-only, and cannot Post (or
even Insert/Edit) using the TSQLTable component, we should use the
ClientDataSet (which can utilize the DataSetProvider to connect back to
the original dbExpress database).
The method that we should call is ApplyUpdates, so we can drop a TButton
on the Form and set its Caption property to Apply Updates.
Inside the OnClick event handler for the Apply Updates button we should write one line of code:
procedure TForm1.Button1Click(Sender: TObject);
begin
ClientDataSet1.ApplyUpdates(0)
end;
This will send all pending updates (inside the ClientDataSet) back to
the original dbExpress database - sending a so-called reconcile error
back if an update error has occurred (such as a record which was already
changed by another user), in which case we must respond to the
OnReconcileError event of the ClientDataSet, see the DataSnap section for more details.
Auto Update
Although the presented solution will work, you cannot realistically
expect your customers and endusers to (remember to) click on the Apply
Updates button when they want to save their work.
We can decide to make it an automatic (apply) update, by responding to
the OnAfterPost event handler of the ClientDataSet component, and
calling ApplyUpdates after each implicit or explicit Post event:
procedure TForm1.ClientDataSet1AfterPost(DataSet: TDataSet);
begin
(DataSet as TClientDataSet).ApplyUpdates(0)
end;
Although this may feel good already, there are still situations we need
to consider: for example deleting records.
There is no Post event after you delete a record, so the OnAfterDelete
event handler should also be implemented, calling the same ApplyUpdates
method.
And finally, when you close your application just after your last change
but before the post (for example if you changed an editfield, but
didn't actually posted the change, yet), then you may want to post this
change.
This means that you need to call the ClientDataSet.ApplyUpdates(0) in
the OnDestroy or OnClose event hander of your data module, form or frame
(or whatever container you are using for your ClientDataSet).
3. ClientDataSet and DataSnap
The third solution (where we spend most of our time) positions the
ClientDataSet as client-side 'briefcase' in a DataSnap multi-tier
application.
This means that we can disconnect the client application, storing the
data locally in a physical briefcase (in the MyBase-format).
We can always reload the data from the briefcase and continue to work
locally.
Until the time when we're back and re-connected to the DataSnap server,
in which case we can send our changes and updates back to the server
(called applying updates).
Note that we do not focus on the server-side issues of DataSnap, nor on
the DataSnap communication protocols, but on the ClientDataSet and its
abilities to connect to local or remote Providers and apply the updates
to the remote middle-ware server.
Finally, applied updates can result in reconcile errors (when one or
more users have already changed fields or records that conflict with our
intended update), and we see how we can detect and respond on these
errors (using the standard reconcile error dialog provided by Borland as
well as a manual implementation).
Multitier Database Architecture
Using a multitier database architecture, you can partition applications
in a way so you can access data on a (database) server without having a
full set of database tools on your local machine.
It also allows you to centralize business rules and processes and
distribute the processing load throughout the network.
DataSnap supports a three-tier technology, which in its classic form
consists of the following:
- A database server on one (server) machine
- An application server on a second (middle-tier) machine
- A thin client on a third (client) machine
The server would be a tool such as InterBase, Oracle, MS SQL server, and so on.
The application server and the thin client would be built in Delphi (or Kylix or C++Builder).
The application server would contain the business rules and the tools for manipulating the data.
The client would do nothing more than present the data to the user for viewing and editing.
What Is DataSnap?
DataSnap is based on technology that allows you to package datasets and
send them across the network as parameters to remote method calls.
It includes technology for converting a dataset into a Variant or XML
package on the server side, and then unbundling the dataset on the
client and displaying it to the user in a grid via the aid of the
TClientDataSet or TInetXPageProducer components.
Seen from a slightly different angle, DataSnap is a technology for
moving a dataset descendant from a TTable or TQuery object on a server
to a TClientDataSet object on a client.
TClientDataSet looks, acts, and feels exactly like a TTable or TQuery
component, except that it doesn't have to be attached to the BDE or any
other database driver for that matter - apart from the DataSnap
middle-ware DLL itself (which is still called MIDAS.DLL by the way).
In this particular case, the TClientDataSet gets its data from unpacking
the variant that it retrieves from the server.
DataSnap allows you to use all the standard Delphi components including
database tools in your client-side applications, but the client side is a
true thin-client: it does not have to include or link any database
drivers apart from the MIDAS.DLL itself (you can even embed the
MIDAS.DLL in the client executable).
DataSnap Clients and Servers
So far the theory.
The best way to understand what DataSnap is and how DataSnap works is to
actually build a DataSnap application, consisting of a DataSnap Client
and DataSnap Server.
Usually, I start with the DataSnap Server, to encapsulate and export the
datasets.
Having a server, the next step is to build a DataSnap Client that
connects to this server and displays the data in some way.
DataSnap Server
To build your first DataSnap Server you need to start with a new empty
application with File | New | Application.
The fact that the mainform of this application is shown actually ensures
that the DataSnap Server will remain loaded (the message loop of the
main form keeps the DataSnap Server alive).
The Caption property of the main form is set to "Dr.Bob's Delphi
Clinic".
However, to be able to easily identify the DataSnap Server, I always
drop a TLabel on the mainform, set its Font property to something that's
big and readable (like Comic Sans MS size 24) and set the Caption
property of the TLabel component to the name of the DataSnap Server ("My
First DataSnap Server" in this case).
To turn a regular application into a middle-ware database server, you
have to add a Remote Data Module to it.
This special data module can be found on the Multitier tab of the Object
Repository, so do File | New | Other and go to the Multitier tab, which
shows several CORBA Wizards, a Remote Data Module and a Transactional
Data Module.
The latter can be used with MTS (Microsoft Transaction Server) before
Windows 2000 or COM+ in Windows 2000 and later, and won't be covered
here.
It's the "normal" Remote Data Module that you need to select to create
your first Simple DataSnap Server.
When you select the Remote Data Module icon and click on the OK-button,
the New Remote Data Module Object Wizard is started.
There are a few options you must specify.
First of all, the CoClass Name, which is the name of the internal class.
This must be a name that you can remember later, so I'll recommend that you use SimpleRemoteDataModule at this time.
Leave Instancing set to Multiple Instance, so your DataSnap Server can contain multiple instances of the Remote Data Module.
Now you can press OK to generate the Remote Data Module.
DataSnap Remote Data Module
The result is a remote data module which looks very much like a regular
data module.
Visually, there's no difference, and if you plan to use the BDE, then
you can begin by treating it like a regular data module by dropping a
TSession component and setting the AutoSessionName property to true
(remember that you need to do this when using the Apartment Threading
Model).
Once you have a TSession component, you can add other components.
For example, you can drop a TTable component and set its name to
TableCustomer.
Set its DatabaseName property to DBDEMOS and open the drop-down combobox
for the TableName property to select the customer.db table.
So far for the regular data module.
Now it's time to look at the remote aspects of this (remote) data
module.
Go to the Data Access tab of the Component Palette.
Here you'll find a TDataSetProvider component.
This component is the key to exporting datasets from a remote data
module to the outside world (more specifically: to DataSnap client
applications).
Drop the TDataSetProvider component on the Remote Data Module, and
assign its DataSet property to TableCustomer.
This means that the TDataSetProvider will "provide" or export the
TableCustomer to a DataSnap client application that connects to it (one
that you will build in the following section).
An important property of TDataSetProvider is the Exported property,
which is set to true to indicate that the TableCustomer is exported.
You can set this property to false to "hide" the fact that TableCustomer
is exported from the Remote Data Module, so clients cannot connect to
it.
This can be useful for example in a 24x7 running server where you need
to make a backup of certain tables, and must ensure that nobody is
working on them during the backup.
With the Exported property set to false, nobody can make a connection to
them anymore (until you set it to true again, of course).
DataSnap Server Compilation
Basically, this is all that it takes to create a first Simple DataSnap
Server.
The only things that's left for you is to save the project.
I've put the main form in file ServerMainForm.pas, the Remote Data
Module will be placed in file RDataMod.pas, and I've put the project
itself in SimpleDataSnapServer.dpr.
After the project is saved you need to compile and run it.
Running the DataSnap Server - which shows only the main form, of course -
will register it (inside the Windows Registry), so any DataSnap Client
can find it in order to connect to it.
If you ever want to move the DataSnap Server to another directory (on
the same machine), you only need to move it and immediately run it
again, so it re-registers itself for that new location.
This is a very convenient way of managing DataSnap Server applications.
So far, you haven't written a single line of Object Pascal code for the Simple DataSnap Server.
Let's see what it takes to write a DataSnap Client to connect to it.
DataSnap Client
There are a number of different DataSnap Clients that you can develop.
Regular Windows (GUI) applications, ActiveForms and even Web Server
applications (using Web Broker or InternetExpress).
In fact, just about everything can act as a DataSnap Client, as you'll
see in a moment.
For now, you'll create a simple regular Windows application that will
act as the first Simple DataSnap Client to connect to the Simple
DataSnap Server of the previous section.
At this stage, you should not be trying to run the client and the server
on separate machines.
Instead, get everything up and running on one machine, and then later
you can distribute the application on the network.
Do a File | New | Application to start a new Delphi Application.
At this time, you may decide to add a data module to it (using File |
New and selecting a Data Module from the Object Repository.
In order to avoid unnecessary screenshots in this paper, I've skipped
the data module, and use the main form to drop my non-visual (DataSnap)
components as well as my normal visual components.
Before anything else, your DataSnap Client must make a connection with
the DataSnap Server application.
This connection can be made using a number of different protocols, such
as (D)COM, TCP/IP (sockets) and HTTP.
The components that implement these connection protocols are
respectively TDCOMConnection, TSocketConnection, TWebConnection and
TCorbaConnection on the DataSnap tab, and the TSoapConnection component
on the WebServices tab.
For the first Simple DataSnap Client, you'll use the TDCOMConnection
component, so drop one from the DataSnap tab onto the main form of your
DataSnap Client.
The TDCOMConnection component has a property called ServerName which
holds the name of the DataSnap Server you want to connect to.
In fact, if you open the drop-down combobox for the RemoteServer
property in the Object Inspector, you'll see a list of all registered
DataSnap servers on your local machine.
In your case, this list might only include one item (namely the
SimpleDataSnapServer.SimpleRemoteDataModule), but all MIDAS 3 and
DataSnap Servers that are registered will end up in this list
eventually.
The names in this list all consist of two parts: the part before the dot
denotes the application name, the part after the dot denotes the Remote
Data Module name.
So, in the current case, you select the SimpleRemoteDataModule of the
SimpleDataSnapServer application.
Once you've selected this RemoteServer, you'll notice that the
ServerGUID property of the TDCOMConnection component also gets a value,
as found in the registry.
Developers with a real good memory are free to type in the ServerGUID
property here in order to automatically get the corresponding
RemoteServer name.
The fun really starts when you double-click on the Connected property of
the TDCOMConnection component, which will toggle this property value
from false to true.
To actually make the connection, the DataSnap Server must be executed,
which results in the pop-up of the main form of the Simple DataSnap
Server that you created in last section.
ClientDataSets
Double-click again on the Connected property of the TDCOMConnection
component to close down the DataSnap Server.
Now that you've seen you can connect to it, it's time to import some of
the datasets that are exported by (the TDataSetProvider component on)
the remote data module.
Drop a TClientDataSet component from the Data Access tab on the main
form, and connect its RemoteServer property to the TDCOMConnection
component.
The TClientDataSet component will obtain its data from the DataSnap
Server.
You now need to specify which provider to use - in other words, from
which TDataSetProvider you want to import the dataset into the
TClientDataSet component.
This can be done with the ProviderName property of the TClientDataSet
component.
Just open the drop-down combobox and you'll see a list of all available
provider names (i.e. all TDataSetProvider components on the Remote Data
Module that have their Exported property set to true at this time).
In this case, there is only one: the only TDataSetProvider component
that you used on the Simple DataSnap Server of last section, so just
select that one.
Before you picked a value for the ProviderName property, you closed down
the Simple DataSnap Server.
However, when you opened up the drop-down combobox to list all available
TDataSetProvider components on the Remote Data Module that currently
have their Exported property set to true, there is only one way (for
Delphi and the Object Inspector) to know exactly which of these
providers are available - by asking the Simple DataSnap Server (more
specifically, by actively looking at the Remote Data Module and finding
out which of the available TDataSetProvider components have their
Exported property set to true.
And since the Simple DataSnap Server was down, it has to be started
again in order to present this list to you in the Object Inspector.
As a result, the moment you drop-down the combobox of the ProviderName
property, the Simple DataSnap Server will be started again.
Once you've selected the RemoteServer and ProviderName, it's time to
open (or activate) the TClientDataSet.
You can do this by setting the Active property of the TClientDataSet
component to true.
At that time, the Simple DataSnap Server is feeding data from the
TableCustomer table via the TDataSetProvider component and a COM
connection to the TDCOMConnection component which routes it to the
TClientDataSet component on your Simple DataSnap Client.
Ready to be used...
And now, you can drop a TDataSource component and move to the Data
Controls tab of the Component Palette and drop one or more data-aware
controls.
To keep the example simple, just drop a TDBGrid component.
Connect the DataSet property of the TDataSource component to the
TClientDataSet, and connect the DataSource property of the TDBGrid
component to the TDataSource.
Since the TClientDataSet component was just activated, you should now
see "live" data at design time, provided by the Simple DataSnap Server.
You are almost ready.
First give the Caption property of the main form a useful name (like
"Simple DataSnap Client").
Then save your work.
Put the main form in file ClientMainForm.pas and call the project
SimpleDataSnapClient.
Then, you're ready to compile and run the Simple DataSnap Client.
Again, you haven't written a single line of Object Pascal code (but rest
assured, that will change soon enough in the upcoming sections).
BriefCase Model
When you run the Simple DataSnap Client, you see the entire
CustomerTable data inside the grid.
You can browse through it, change field values, even enter new records
or delete records.
However, once you close the application, all changes are gone, and
you're back at the original dataset inside the Delphi IDE again.
No matter how hard you try, the changes that you make to the visual data
seem to affect the data inside the (local) TClientDataSet only, and not
the (remote) actual TableCustomer.
What you experience here is actually a feature of the so-called briefcase model.
Using this briefcase model, you can disconnect the client from the network and still access the data.
It works as follows:
- Save a remote dataset to disk, shut down your machine, and disconnect from the network.
You can then boot up again and edit your (local) data without connecting to the network.
- When you get back to the network, you can reconnect and update the database.
A special mechanism is provided for being notified about database errors and resolving any conflicts that might occur.
For instance, if two people edited the same record, then you will be notified of the fact and given options to resolve it.
The point is that you don't have to actually be able to reach the server at all times to be able to work with your data.
This capability is ideal for laptop users or for sites where you want to keep database traffic to a minimum.
Now then, you've already experienced that (apparently) your Simple
DataSnap Client works on the local data inside your TClientDataSet
component only.
It appears you can even save the data to a local file, and load it back
in again.
To save the current content of a TClientDataSet, drop a TButton on the
main form, set the Name property to ButtonSave, set the Caption to Save,
and write the following code for the OnClick event handler:
procedure TClientForm.ButtonSaveClick(Sender: TObject);
begin
ClientDataSet1.SaveToFile('customer.cds')
end;
This saves all records from the TClientDataSet in a file called
customer.cds in the current directory.
By the way, cds stands for ClientDataSet, but you can use your own file
and extension names, of course.
Note the dfBinary flag which is passed as second argument to the
SaveToFile method of the TClientDataSet.
This value indicates that I wish to save the data in binary - Borland
propriety - format.
Alternately, I could specify to save the data in XML format, passing the
dfXML value.
An XML file will be much larger (14K vs. 7K for the entire TableCustomer
data), but has the advantage that it can potentially be used by other
applications as well.
I'll just stick to the smaller (and more efficient) binary format.
Similarly, to implement the functionality that you can load the
customer.cds file again into your TClientDataSet component, you need to
drop another TButton component, set its Name property to ButtonLoad, set
the Caption to Load, and write the following Object Pascal code for the
OnClick event handler:
procedure TClientForm.ButtonLoadClick(Sender: TObject);
begin
ClientDataSet1.LoadFromFile('customer.cds')
end;
Note that the LoadFromFile method of the TClientDataSet component does
not need a second argument; it's obviously smart enough to determine
whether it's reading a binary or an XML file.
And while the binary file can probably only be generated by another
TClientDataSet component, the XML file could actually have been produced
by an entirely other application.
Armed with these two buttons, you can now (locally) save the changes to
your data, and even reload those changes once you stop and start the
Simple DataSnap Client application again.
In order to control the fact whether or not the TClientDataSet component
is "live" connected to the DataSnap Server, you can drop a third
TButton component on the form which toggles the Active property of the
TClientDataSet component.
Set the Name property of this TButton to ButtonConnect, and give the
Caption property the value Connect.
Now, write the following code for the OnClick event handler:
procedure TClientForm.ButtonConnectClick(Sender: TObject);
begin
if ClientDataSet1.Active then // close and disconnect
begin
ClientDataSet1.Close;
DCOMConnection1.Close;
end
else // open (will automatically connect)
begin
// DCOMConnection1.Open;
ClientDataSet1.Open;
end
end;
Note that in order to close the connection you actually have to Close
the TClientDataSet component and Close the TDCOMConnection as well,
while in order to open the connection you only need to Open the
TClientDataSet component, which will implicitly Open the TDCOMConnection
as well.
Finally, there's one more thing you really need to do: make sure the
TDCOMConnection and TClientDataSet components are not connected to the
SimpleDataSnapServer at design-time.
Otherwise, whenever you re-open your SimpleDataSnapClient project in the
Delphi IDE again, it will need to make a connection to the
SimpleDataSnapServer - loading that DataSnap Server.
And when - for one reason or another - the SimpleDataSnapServer is not
found on your machine, you will have a hard time loading the
SimpleDataSnapClient project.
So I always make sure they are not connected at design-time.
In order to do so, you have to assign false to the Connected property of
the TDCOMConnection component (which will unload the main form of the
SimpleDataSnapServer) and false to the Active property of the
TClientDataSet component (which results in the fact that you won't see
any data at design-time anymore).
Let me take a moment to discuss the whole process of clients timing out
when they can't talk to their server.
If you try to talk to DCOM server but can't reach it, the system will
not immediately give up the search.
Instead, it can keep trying for a set period of time that rarely exceeds
two minutes.
During those two minutes, however, the application will be busy and will
appear to be locked up.
If the application is loaded into the IDE, then all of Delphi will
appear to be locked up.
You can have this problem when you do nothing more than attempt to set
the Connected property of the TDCOMConnection component to true.
Now, when you recompile and run your SimpleDataSnapClient, it will show
up with no data inside the TDBGrid component.
And this is the time where you can click on the Connect button in order
to connect to the SimpleDataSnapServer and obtain all records (from the
database server).
However, there are times (for example when you are "on the road" or
simply not connected to the machine that runs the SimpleDataSnapServer),
when you cannot connect to the SimpleDataSnapServer.
In those cases, you can click on the Load button instead, and work on
the local copy of the records.
Note that this local copy is the one that you last saved, and is only
updated when you click on the Save button to write the entire contents
of the TClientDataSet component to disk.
ApplyUpdates
It's nice to be able to Connect or Load up a local dataset and Save it
to disk again.
But how do you ever apply your updates to the actual (remote) database
again? This can be done using the ApplyUpdates method of the
TClientDataSet component.
Drop a fourth button on the SimpleDataSnapClient main form, set its Name
property to ButtonApplyUpdates and the Caption property to "Apply
Updates".
Like the Save button, this button should only be enabled when the
TClientDataSet component actually contains some data (I leave that code
as an exercise for the readers - contact me if you're having problems
with it).
The OnClick event handler of the Apply button should get the following simple line of code:
procedure TClientForm.ButtonApplyClick(Sender: TObject);
begin
if ClientDataSet1.ChangeCount > 0 then
ClientDataSet1.ApplyUpdates(0)
end;
The ApplyUpdates method of the TClientDataSet component has one
argument: the maximum number of errors that it will "allow" before
stopping with applying (more) updates.
With a single SimpleDataSnapClient connected to the
SimpleDataSnapServer, you will never encounter any problems, so feel
free to run your SimpleDataSnapClient now.
Click on the Connect button to connect to (and load) the
SimpleDataSnapServer, and use the Save and Load buttons to store and
read the contents of the TClientDataSet component to and from disk.
You can even remove your machine from the network and work on your local
data for a significant amount of time, which is exactly the idea behind
the briefcase model (your laptop being the briefcase).
Any changes you will make to your local copy will remain visible, and
you can apply the changes back to the remote database with a click on
the Apply Updates button - once you've reconnected to the network with
the SimpleDataSnapServer.
Error Handling
So what if two clients, both using the BriefCase Model, connect to the
Simple DataSnap Server, obtain the entire TableCustomer and both make
some changes to the first record.
According to what you've build so far, both clients could then send the
updated record back to the DataSnap Server using the ApplyUpdates method
of their TClientDataSet component.
If both pass zero as value for the "MaxErrors" argument of ApplyUpdates,
then the second one to attempt the update will be stopped.
The second client could pass a numerical value bigger than zero to
indicate a fixed number of errors/conflicts that are allowed to occur
before the update is stopped.
However, even if the second client passed -1 as argument (to indicate
that it should continue updating no matter how many errors occur), it
will never update the records that have been changed by the previous
client.
In other words: you need to perform some reconcile actions to handle
updates on already-updated records and fields.
Fortunately, Delphi contains a very useful dialog especially written for
this purpose.
And whenever you need to do some error reconciliation, you should
consider adding this dialog to your DataSnap Client application (or
write one yourself, but at least do something about it).
To use the one available in Delphi, just do File | New | Other, go to
the Dialogs tab of the Object Repository and select the Reconcile Error
Dialog icon.
Once you select this icon and click on OK, a new unit is added to your
SimpleDataSnapClient project.
This unit contains the definition and implementation of the Update Error
dialog that can be used to resolve database update errors.
Once this unit is added to your SimpleDataSnapProject, there is
something very important you have to check.
First save your work (put the new unit in file ErrorDialog.pas).
Now, unless you've already unchecked the option to "Auto create forms"
inside the Designer tab of the Tools | Environment Options dialog, you
need to make sure that the TReconcileErrorForm is not one of the forms
that are autocreated by your application - see the Forms tab of the
Project | Options dialog.
In the Forms tab of the Project | Options dialog, you'll find a list of
Auto-create forms and a list of Available forms.
Just check to make sure the ReconcileErrorForm is not on the list of
Auto-create forms.
An instance of the ReconcileErrorForm will be created dynamically,
on-the-fly, when it is needed.
So when or how do you use this special ReconcileErrorForm? Well, it's
actually very simple.
For every record for which the update did not succeed (for whatever
reason), the OnReconcileError event handler of the TClientDataSet
component is called.
This event handler of TClientDataSet is defined as follows:
procedure TClientForm.ClientDataSet1ReconcileError(DataSet: TClientDataSet;
E: EReconcileError; UpdateKind: TUpdateKind;
var Action: TReconcileAction);
begin
end;
This is an event handler with four arguments: first of all the
TClientDataSet component that raised the error, second a specific
ReconcileError that contains a message about the cause of the error
condition, third the UpdateKind (insert, delete or modify) that
generated the error and finally as fourth argument the Action that you
feel should be taken.
As Action, you can return the following possible enum values (the order is based upon their actual enum values):
- raSkip - do not update this record, but leave the unapplied changes in the change log.
Ready to try again next time.
- raAbort - abort the entire reconcile handling; no more records will be passed to the OnReconcileError event handler.
- raMerge - merge the updated record with the current record in
the (remote) database, only changing (remote) field values if they
changed on your side.
- raCorrect - replace the updated record with a corrected value
of the record that you made in the event handler (or inside the
ReconcileErrorDialog.
This is the option in which user intervention (i.e. typing) is required.
- raCancel - undo all changes inside this record, turning it back into the original (local) record you had.
- raRefresh - undo all changes inside this record, but reloading
the record values from the current (remote) database (and not from the
original local record you had).
The good thing about the ReconcileErrorForm is that you don't really
need to concern yourself with all this.
You only need to do two things.
First, you need to include the ErrorDialog unit inside the
SimpleDataSnapClient main form definition.
Click on the ClientMainForm and do File | Use Unit to get the Use Unit
dialog.
With the ClientMainForm as your current unit, the Use Unit dialog will
list the only other available unit, which is the ErrorDialog.
Just select it and click on OK.
The second thing you need to do is to write one line of code in the
OnReconcileError event handler in order to call the HandleReconcileError
function from the ErrorDialog unit (that you just added to your
ClientMainForm import list).
The HandleReconcileError function has the same four arguments as the
OnReconcileError event handler (not a real coincidence, of course), so
it's a matter of passing arguments from one to another, nothing more and
nothing less.
So, the OnReconcileError event handler of the TClientDataSet component
can be coded as follows:
procedure TClientForm.ClientDataSet1ReconcileError(DataSet: TClientDataSet;
E: EReconcileError; UpdateKind: TUpdateKind;
var Action: TReconcileAction);
begin
Action := HandleReconcileError(DataSet, UpdateKind, E)
end;
Demonstrating Reconcile Errors
The big question now is: how does it all work in practice? In order to
test it, you obviously need two (or more) SimpleDataSnapClient
applications running simultaneously.
For a complete test using the current SimpleDataSnapClient and
SimpleDataSnapServer applications, you need to perform the following
steps:
- Start the first SimpleDataSnapClient, and click on the Connect button (the SimpleDataSnapServer will now be loaded as well).
- Start the second SimpleDataSnapClient and click on the Connect button.
Data will be obtained from the same SimpleDataSnapServer that's already running.
- Using the first SimpleDataSnapClient, change the field "Company" for the first record.
- Using the second SimpleDataSnapClient, also change the field
"Company" for the first record (make sure you don't change it to the
same value as you did in the previous step using the first
SimpleDataSnapClient).
- Click on the "Apply Updates" button of the first SimpleDataSnapClient.
All updates will be applied without any problems.
- Click on the "Apply Updates" button of the second
SimpleDataSnapClient.
This time, one or more errors will occur, because the first record has
its "Company" field value changed (by the first SimpleDataSnapClient),
and for this and possibly more conflicting records, the OnReconcileError
event handler is called.
- Inside the Update Error dialog, you can now experiment with the
reconcile Actions (skip, abort, merge, correct, cancel and refresh) to
get a good feeling of what they do.
Pay special attention to the differences between Skip and Cancel, and
the differences between Correct, Refresh and Merge.
Skip moves on to the next record, skipping the requested update (for the
time being).
The unapplied change will remain in the change log.
Cancel also skips the requested update, but it cancels all further
updates (in the same update packet).
The current update request is skipped in both cases, but Skip continues
with other update requests, and Cancel cancels the entire ApplyUpdate
request.
Refresh just forgets all updates you made to the record and refreshes
the record with the current value from the server database.
Merge tries to merge the update record with the record on the server,
placing your changes inside the server record.
Refresh and Merge will not process the change request any further, so
the records are synchronized after Refresh and Merge (while the change
request can still be redone after a Skip or Cancel).
Correct, the most powerful option, actually gives you the option of customizing the update record inside the event handler.
For this you need to write some code or enter the values in the dialog yourself.
Summary
In this paper, I have described you how TClientDataSet can be used as
stand-alone (in-memory) database table, as well as the use incombination
with dbExpress (for 2-tier) and DataSnap (for 3-tier) as database
cache, providing the so-called Briefcase Model.
With the frozen BDE, the importance of the ClientDataSet component
continues to grow in Delphi, Kylix and C++Builder applications!
For more information, see Dr.Bob Examines #63 on C++Builder Data Access Technologies and #58 on Delphi Database Development, as well as my BorCon 2004 paper on Data Access Techniques with ClientDataSets and the BorCon 2003 paper on Introduction to ClientDataSet and dbExpress.
|