REST easy with kbmMW #5 – Logging

Following up on the previous blog posts about how easily to create a REST server with kbmMW, I today want to write a little bit about logging.

kbmMW contains a quite sophisticated logging system, which lets the developer log various types of information whenever the developer needs it, and at runtime lets the administrator decide what type of log to react on and how.

In addition the log can be output in a file, in the system log (OS dependant), or be sent to a remote computer for storage. In fact all the above methods can coexist at once.

What’s the purpose of logging?

Well. There can be multiple purposes, amongst others:

  • For debugging while developing
  • For debugging after deployment
  • For keeping track of resources
  • For keeping track of usage (perhaps even relates to later invoicing)
  • For proving reasons for user complaints
  • Of security reasons to track who is doing what

As you can tell, there seems to be various log requirements for various stages of the lifetime of the application:

  • During development
  • During usage
  • Early warning
  • Post incident investigation

A good log system should imo handle all the above scenarios, while making it simple to use for the developer, and allow the administrator to tune on the amount of information needed.

kbmMW’s log system handles all these scenarios, and can be late fine tuned for the required log level.

In addition the log system should be able to output the log in relevant formats, that match the application’s purpose.

Web server applications, might want to output some log data in a format generally accepted by web servers, and thus also by web server log file analyzer software, while other server applications may have other requirements for output.

kbmMW supports several output formats, and also allows adding additional formats, without having to make changes in the developer’s logging statements.

So let us get on with it.

First add the kbmMWLog unit to the units in which you expect to do some logging.

In our case, we have the units Unit7 (main form unit), Unit8 (Smart service unit… the actual REST business code) and Unit9 (a defined sharable TContact object).

It makes sense to add support for logging in Unit7 and Unit8. In Unit7 it would look similar to this:

interface

uses
 Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
 Vcl.Controls, Vcl.Forms, Vcl.Dialogs, kbmMWCustomTransport, kbmMWServer,
 kbmMWTCPIPIndyServerTransport, kbmMWRESTTransStream,
 kbmMWCustomConnectionPool, kbmMWCustomSQLMetaData, kbmMWSQLiteMetaData,
 kbmMWSQLite, kbmMWORM, IdBaseComponent, IdComponent, IdServerIOHandler, IdSSL,
 IdSSLOpenSSL, IdContext, kbmMWSecurity, kbmMWLog;

And in Unit8 we have also added kbmMWLog to the uses clause.

By simply adding this unit, we can already log by calling one of the methods of the public default available Log instance. Eg.

Log.Debug('some debug information');
Log.Info('2 + 2 = %d',[2+2]);

kbmMW’s log system supports these easy access methods:

  • Debug (typically used during development purposes),
  • Info (inform about some non critical and non error like information)
  • Warn (inform about some non critical anormal situation)
  • Error (inform about some error, like an exception or something else which still allow the application to continue to operate)
  • Fatal (inform about an error of such magnitude that the application no longer can run).
  • Audit (inform about some information that you want to be used as evidence in a post analysis scenario).

They in turn calls a number of generic TkbmMWLog.Log method which takes arguments for log type, severity, timestamps and much more.

You can ask kbmMW to log content of streams, of memory buffers, XML and JSON documents, byte arrays, and you can even ask kbmMW to produce a stack trace along with your log (not currently supported on NextGen platforms).

In our simple REST server, we might want to log whenever a user logs in, when they are logged out, when a function is called, and when an exception happens.

To intercept the login situation, we will write some event handlers for the OnLoginSuccess and OnLoginFailed event on the TkbmMWAuthorizationManager instance we have on Unit7.

procedure TForm7.kbmMWAuthorizationManager1LoginFail(Sender: TObject;
 const AActorName, ARoleName, AMessage: string);
begin
 Log.Warn('Failed login attempt as %s with role %s. %s',[AActorName,ARoleName,AMessage]);
end;

procedure TForm7.kbmMWAuthorizationManager1LoginSuccess(Sender: TObject;
 const AActorName, ARoleName: string; const AActor: TkbmMWAuthorizationActor;
 const ARole: TkbmMWAuthorizationRole);
begin
 Log.Info('Logged in as %s with role %s',[AActorName,ARoleName]);
end;

It makes sense to log a successful login as an information, while an unsuccessful login is logged as a warning. If it happens often, it could be malicious login attempts, so warnings ought to be looked after.

And we might also want to log a logout of a user. The logout may happen automatically due to the user being idle for too long. Refer to the previous blog post for more information.

We might also want to log what calls are made by logged in users.

This can be done in many ways and many places. You could choose to do it within your business logic code in the smart service in Unit8, which makes sense if you want to log some more specific information about the call.

But if you just want to log successful and failed calls, then it’s easy to do so using the OnServeResponse event of the TkbmMWServer instance in Unit7.

As long as the request is formatted correctly and thus served through the TkbmMWServer, it will be attempted to be executed, and a response sent back to the caller.

The execution may succeed or it may fail, but in all cases the OnServeResponse event will be triggered.

procedure TForm7.kbmMWServer1ServeResponse(Sender: TObject;
 OutStream: IkbmMWCustomResponseTransportStream; Service: TkbmMWCustomService;
 ClientIdent: TkbmMWClientIdentity; Args: TkbmMWVariantList);
begin
 if OutStream.IsOK then
   Log.Info('Successfully called %s on service %s',[ClientIdent.Func,ClientIdent.ServiceName])
 else
   Log.Error('An error "%s" happened while serving request for %s on %s',[ClientIdent.Func,ClientIdent.ServiceName,OutStream.StatusText]);
end;

Now we intercepts and logs at strategic places in our code, and in fact the logging is already working. But the log output is currently only placed on the system log, which on Windows is interpreted as the debugger.

We need to have our log output to a file, preferably with nice chunking when the file reach a certain size.

The responsibility of the actual output, is the log manager. There are a number of log managers included with kbmMW:

  • TkbmMWStreamLogManager – Sends log to a TStream descendant.
  • TkbmMWLocalFileLogManager – Sends log to a file.
  • TkbmMWSystemLogManager – Sends log to system log (depends on OS).
  • TkbmMWStringsLogManager – Sends log to a TStrings descendant.
  • TkbmMWProxyLogManager – Proxies log to another log manager.
  • TkbmMWTeeLogManager – Sends log to a number of other log managers.
  • TkbmMWNullLogManager – Sends log to the bit graveyard.

If you have kbmMW Enterprise Edition and thus also have access to the WIB (Wide Information Bus) publish/subscribe transports, you have a couple of additional log managers available for remote logging:

  • TkbmMWClientLogManager – Publishes logs via the WIB
  • TkbmMWServerLogManager – Subscribes for logs on the WIB, and forwards those through other log managers.

You can make your own log manager by descending from TkbmMWCustomLogManager and implementing the IkbmMWLogManager interface.

To use a different log manager than the default system log manager, you simply create an instance of the log manager you want to use and assign it to the TkbmMWLog.Log.LogManager property. Eg.

Log.LogManager:=TkbmMWLocalFileLogManager.Create('c:\temp\mylogfile.log');

However to set specific settings on the log manager, it is better to instantiate a variable with it, set its properties and then later assign that variable to the Log.LogManager property.

An even easier way, is to use one of the Log.Output….. methods, which easily creates relevant log managers for you with settings that usually are good for most circumstances. Eg.

Log.OutputToDefaultAndFile('c:\temp\mylogfile.log');

This will in fact create 3 log managers, a system log manager, a file log manager and a tee log manager and automatically hooks them all up.

In our case we just want to output to a file, so let us stick with the TkbmMWLocalFileLogManager. So we will simply create an instance and assign it to the Log.LogManager as shown above.

Now all the log will be output to the file, and the file will automatically be backed up and a new created when it reaches 1MB size. Backup naming and size etc. are all configurable on the TkbmMWLocalFileLogManager instance.

You can control which fields are output via the Log.LogManager.LogFormatter property. It is default a TkbmMWStandardLogFormatter. kbmMW also supports a TkbmMWSimpleLogFormatter which only outputs date/time, type and the actual log string.

The standard log formatter also outputs data type, process and thread information and binary data (usually converted to either Base64 or hexdump (pretty) format).

There are much more to logging. We didn’t touch the fact that the log system can handle separate log files for auditing and other logging, and that you can set filtering on each log manager so that particular log manager only logs certain log types or log levels or data types.

Happy logging.

ANN: kbmMW Professional and Enterprise Edition v. 5.02.00 released!

We are happy to announce v5.02 of our popular middleware for Delphi and
C++Builder.

If you like kbmMW, please let others know! Share the word!

We strive hard to ensure kbmMW continues to set the bar for what an n-tier product must be capable of in the real world!

Keywords for this release:

  • Many ORM improvements
  • New! CRON compatible scheduler support
  • Synchronous encryption improvements
  • AMQP improvements
  • Many other improvements and bugfixes for reported bugs

Please look in the end of this post for a detailed change list.

Professional and Enterprise Edition is available for all with a current
active SAU. If your SAU has run out, please visit our shop to extend it with another
12 months.

CodeGear Edition is available for free, but only supports a specific Delphi/Win32 SKU, contains a limited feature set and do not include source.

Please visit https://portal.components4developers.com to download.

—-

kbmMW is the premiere n-tier product for Delphi, C++Builder and FPC on .Net, Win32, Win64, Linux, Java, PHP, Android, IOS, embedded devices, websites, mainframes and more.

Please visit http://www.components4developers.com for more information about kbmMW.

—-

Components4Developers is a company established in 1999 with the purpose of providing high quality development tools for developers and enterprises. The primary focus is on SOA, EAI and systems integration via our flagship product kbmMW.

kbmMW is a portable, highly scalable, high end application server and enterprise architecture integration (EAI) development framework for Win32, ..Net and Linux with clients residing on Win32, .Net, Linux, Unix, Mainframes, Minis, Embedded and many other places. It is currently used as the backbone in hundreds of central systems, in
hospitals, courts, private, industries, offshore industry, finance, telecom, governements, schools, laboratories, rentals, culture institutions, FDA approved medical devices, military and more.


5.02.00 May 27 2017

Important notes (changes that may break existing code)
======================================================
* Changed Use class in kbmMWSmartUtils.pas. Now it will use TkbmMWAutoValue internally
to store data. Since data stored in TkbmMWAutoValue is reference counted and scoped,
access to the data is slightly different.
Use AsObject to return a reference to the object. Ownership of the object belongs to the TkbmMWAutoValue container.
Use AsMyObject to return a reference to the object and mark it as your object. You will be responsible for freeing it.

New stuff
=========
– Added IkbmMWAutoValue and TkbmMWAutoValue to kbmMWGlobal.pas. They handle scope based object life time handling.
– Changed smart object’s (TkbmMWMarshalledVariantData) to use TkbmMWAutoValue.
– Updated DumpVariant in kbmMWGlobal.pas to dump smart object’s too.
– Added support for TkbmMWTiming on IOS.
– Added support for REST tags anonymousRoot=true/false and pretty=true/false which
can be used to control if resulting objects should be anonymous or contained in a
parent object, and if the result should be prettyformatted or not (default).
Prettyformatting is not implemented on JSON at the time.
– Updated AMQP protocol to default not write shortstrings, unsigned 8bit, unsigned16 bit
unsigned32 bit and unsigned 64 bit. Reason is that although AMQP v. 0.9.1
should support them, industry dont, why most AMQP implementations will not understand those types.
It is possible to uncommnet a number of defines in top of kbmMWAMQP.pas to selectively
enable these types. If they are commented, kbmMW auto propogates the value to the next
sensible type.
– Updated kbmMWAMQP.pas to support copying field tables instead of assigning them.
– Added Safe property to TkbmMWMixerPasswordGen. If set to true, it will not use
digits and characters that can be visually misread (0 vs O etc).
– Added OnMessageProcessingFailed event to TkbmMWCustomSAFClientTransport and
TkbmMWCUstomSAFServerTransport and published in descending classes.
It will be called when message processing failed, for example if kbmMW is
unable to decrypt a message.
– Added support for dynamic arrays in object marshalling.
– Added support for Notify in TkbmMWDateTime and kbmMWNullable. If set (in an ORM scenario)
the client will be notified about the value in that particular field.
– Modified and fixed timezone initialization in kbmMWDateTime.pas.
– Added OutputToDefaultAndFile and OutputToDefaultAndStringsAndFile to TkbmMWLog for
easy setup of outputs.
– Enhanced TkbmMWCustomCrypt to support PassPhraseBytes (which if set, takes precedence over
PassPhrase (string).
– Added OnEncryptKeys, OnDecryptKeys, OnDecryptStatus events to TkbmMWCustomCrypt to allow for
attempting various keys before finally either succeeding or giving up.
This can be valuable in supporting client unique encryption/decryption.
– Added a number of GetDefAs…. methods to TkbmMWONArray and TkbmMWONObject which
returns a default value if the property/index is missing instead of raising an exception.
– Added GlobalIndexNames property to TkbmMWCustomSQLMetaData. If set kbmMW’s SQL rewriter
knows that index names must be database scope unique, instead of only table scope unique.
– Added Init function that accepts a string as salt to TkbmMWCustomHash.
– Added GetDigest to TkbmMWCustomHash, which returns a byte array with the digested hash.
Its an alternative to using Final.
– Added OnDisconnected and OnException events to TkbmMWAMQPClientConnection.
– Added OnConnect, OnDisconnect, OnDisconnected and OnException events to TkbmMWAMQPClient.
– Added mwsloDateTimeIsUTC to TkbmMWSQLiteOption. Determines how to interpret date time values, as local time or as UTC time.
– Added support for boolean parameter values in TkbmMWSQLite.
– Improved marshalling of kbmMWNullable types.
– Added kbmMWSubjectGetType, kbmMWSubjectExtractNodeID and
kbmMWGenerateMessageSubscriptionSubject to kbmMWSubjectUtils.pas
– Added mwrieNotify to TkbmMWRecordInfoEvent in kbmMWCustomDataset.pas
– Added support for TIMESTAMP datatype in SQL datatype deduction.
– Added support for returning an interfaced object from smart services.
– Added field change detection to TkbmMWFieldDefs.
– Improved TkbmMWRTTI.InstantiateValue in kbmMWRTTI.pas.
– Improved kbmMWNullable.
– Changed Use class in kbmMWSmartUtils.pas. Now it will use TkbmMWAutoValue internally
to store data. Since data stored in TkbmMWAutoValue is reference counted and scoped,
access to the data is slightly different.
Use AsObject to return a reference to the object. Ownership of the object belongs to the TkbmMWAutoValue container.
Use AsMyObject to return a reference to the object and mark it as your object. You will be responsible for freeing it.
– Added methods ToDataset, FromDataset, ListFromDataset to TkbmMWSmartClientORM.
Provides an easy way to convert arguments and results to and from datasets.
– Added Cron fluent method to IkbmMWScheduledEvent. It accepts a 5 or 6 part Unix cron value
which defines the interval.
– Added AtYears, AtMonths, AtDays, AtHours, AtMinutes, AtSeconds methods to IkbmMWScheduledEvent
to give an alternative way to provide cron like schedules.
– Added SynchronizedAfterRun and AfterRun methods to IkbmMWScheduledEvent to
provide an anonymous function to be called after the schedule has run.
It is particular valuable on scheduling asynchronous operations via RunNow,
followed up with updating something with the result of the function.
– Added TkbmMWONSchedulerStorage for storing/retrieving schedules in any object notation format.
– Added support for subscribing for raw messages using anonymous function in WIB.
– Added Delete to TkbmMWORM taking primary key values alternative specific field values.
– Added support for many more date formats for ORM data generators.
In addition to LOCAL, UTC and ISO8601, also RFC1123, NCSA, LOCALSINCEEPOCHMS,
UTCSINCEEPOCHMS, LOCALSINCEEPOCH and UTCSINCEEPOCH is supported.
– Generally many additional improvements on ORM.

Fixes
=====
– Fixed default true/false values for TkbmMWSQLiteMetaData.
– Fixed HTTP/REST/AJAX additional incorrect CRLF in output.
– Fixed serious bug in 32 bit random generators (kbmMWRandom.pas).
– Fixed NextGen issues in some parsing routines in kbmMWDateTime.pas.
– Fixed bugs in Query service wizard.
– Fixed some SQL rewriting bugs including adding support for DESCENDING order by.

REST easy with kbmMW #4 – Access management

Building on the previous articles about how to create a REST server using kbmMW, we have now reached the stage where we should consider access management.

What is access management? It’s the “science of who are allowed to do what.

It is obvious that data exists in this world, which should be protected from being read, created or altered by people/processes we have not authorized to do so. Or turned on its head, some data should be protected and be accessible only by people/processes that we trust.

Other data might be left freely available for reading, but never for modifying and so forth.

Fortunately kbmMW have features built in to support us with that.

We start by adding a TkbmMWAuthorizationManager to the main form (Unit7 in the previous posts).2017-05-26 15_07_56-Project6 - RAD Studio 10.1 Berlin - Unit7.png

We can use the authorization manager as is, standalone, but it often makes sense to connect it to the kbmMWServer instance. Thus set the property kbmMWServer1.AuthorizationManager to point on kbmMWAuthorizationManager1.

This way, every call into the application server will checked by the authorization manager for access rights.

The kbmMW authorization manager is an entity which understands the topics:

  • resource
  • actor
  • role
  • authorization
  • constraint
  • login

A resource is basically anything that you want to add some sort of protection for. It can be database related, it can be a specific object, it can be a function or a service that you want to ensure is only handled in ways that you want it to, by people/processes that you have granted access to it. Resources can be grouped in resource trees, where having access to one resource also automatically provides same access to resources underneath that resource.

An actor, is typically a person (or a person’s login credentials), a process or something else that identifies “someone” that want access to your resource’s.

A role is a way to categorize general access patterns. Roles in a library, could be a librarian, an administrator and a loaner. Roles in a bank could be a customer, a teller, a clerk, an administrator and so forth. The idea is that each of the roles will have different access rights to the various resources. Actors usually will be given at least one role. An actor can have different roles, for example depending on how the actor log’s in, or from where.

An authorization is a “license” to operate as an actor or a role on a specific resource. An authorization can be negative, thus specifically denying an actor or role access to specific resources and their subtrees.

A constraint is a limitation to an authorization or to a login. The authorization may only be valid within a specific timeframe, or be allowed to be accessed from specific equipment and such, or the login can only happen during daytime etc.

A login is the match between an actor/password and a login token. When an actor is attempting to be logged in, the system verifies login name, password, requested role and whatever constraints has been defined related to login in. Only when everything has been checked up and a login is allowed, a token is issued, which the actor/user/process will need to send along with every request it makes to the kbmMW based server.

So let us define two roles we want to have access to our REST server. We can choose to name them ‘Reader’ and ‘ReadWriter’, but as kbmMW do not pose any restrictions to naming of roles (nor on actors and resources), we can name them anything as long as the names are unique within their category (roles, actors, resources).

  • Reader
  • ReadWriter

In code we define the roles like this (for example in the OnCreate event of the main form:

 kbmMWAuthorizationManager1.AddRole('READER');
 kbmMWAuthorizationManager1.AddRole('READWRITER');

We also, somehow, need to tell the authorization manager which actors exists so it can match up login attempts with actors.

The simple way is to predefine them to the authorization manager. That can for example also happen in the OnCreate event of the form, or elsewhere before the first access to the server. The actors can be defined from a database or a configuration file or LDAP etc. as needed.

 kbmMWAuthorizationManager1.AddActor('HANS','HANSPASSWORD','READER');
 kbmMWAuthorizationManager1.AddActor('CHRISTINE','CHRISTINEPASSWORD','READWRITER');

This defines two actors with their passwords, and which role they should act as upon login if they do not specifically ask for a different role.

It is possible not to predefine actors, but instead use an event handler to verify their existence in a different system via the OnLogin event of the kbmMWAuthorizationManager1 instance.

procedure TForm7.kbmMWAuthorizationManager1Login(Sender: TObject;
 const AActorName, ARoleName: string; var APassPhrase: string;
 var AActor: TkbmMWAuthorizationActor; var ARole: TkbmMWAuthorizationRole;
 var AMessage: string);
begin
...
end;

An AActorName and the requested role name in ARoleName is provided. Optionally an actor instance may also be provided, if the actorname is known to kbmMW. If not, AActor is nil, and must be created by you if you know about the actor.

ARole may be nil, if it’s an unknown role that is requested. You can choose to define the role on the fly by returning a newly created TkbmMWAuthorizationRole instance. Remember to add any newly created actor or role instances to the kbmMWAuthorizationManagers Actors and Roles list properties before returning.

APassword will contain the password delivered with the login attempt. You are allowed to modify it on the fly (for example to change it to a SHA256 hash, so no human readable passwords are stored in the authorization manager).

If you return nil for AActor or ARole, then it means that the login failed. You can provide an explanation in the AMessage argument if you want.

But let us continue with our simple actor definition for this sample

Now that we have actors and roles defined, the authorization manager is ready to handle login attempts.

There is only one way to login, and that is by calling the Login method of the authorization manager. This can, for example, be called from a new REST function in your REST service.

An alternative is to let kbmMW automatically detect login attempts, and call the Login method for you. To do that, set the Options property of kbmMWAutorizationManager1 to [mwaoAutoLogin].

As you may remember, all requests to the kbmMW server must be accompanied with a Token identifying a valid login. If that token is not available, kbmMW (with mwaoAutoLogin set), is triggered to use whatever username/password passed on from the caller, as data for a login attempt and will return the token back to the called if the login succeeded.

As a REST server is essentially a web server, adhering to the HTTP protocol standards, what happens when kbmMW detects an invalid (or non existing) login, is that kbmMW will raise an EkbmMWAuthException, which in turn (when the call comes via the REST streamformat), will be translated into an HTTP error 401, which is presented to the caller. In fact, if you would raise that exception anywhere within your business code and you do not manage it yourself, it will automatically be forwarded to the caller as a 401.

This will prompt most browsers to present a login dialog, where username/password can be entered, and next call to back to the server, will include that login information. kbmMW will automatically detect this and use it.

So we have actor, role and login in place. Now we need to determine what resources we have. A resource can be anything you want to tag a unique name on.

Most of the time, it makes sense to define REST methods as a resource. This is done very easily in our smart service, where we have the functions for manipulating and retrieving contacts (Unit8). We use the kbmMW_Auth attribute.

[kbmMW_Service('name:MyREST, flags:[listed]')]
[kbmMW_Rest('path:/MyREST')]
TkbmMWCustomSmartService8 = class(TkbmMWCustomSmartService)
 public
  [kbmMW_Auth('role:[READER,READWRITER], grant:true')]
  [kbmMW_Rest('method:get, path:helloworld, anonymousResult:true')]
  [kbmMW_Method]
  function HelloWorld:TMyResult;

  [kbmMW_Auth('role:[READER,READWRITER], grant:true')]
  [kbmMW_Rest('method:get, path:contacts, anonymousResult:true')]
  function GetContacts:TObjectList<TContact>;

  [kbmMW_Auth('role:[READWRITER], grant:true')]
  [kbmMW_Rest('method:put, path:addcontact')]
  function AddContact([kbmMW_Rest('value:"{$name}"')] const AName:string;
    [kbmMW_Rest('value:"{$address}"')] const AAddress:string;
    [kbmMW_Rest('value:"{$zipcode}"')] const AZipCode:string;
    [kbmMW_Rest('value:"{$city}"')] const ACity:string):string; overload;

  [kbmMW_Auth('role:[READWRITER], grant:true')]
  [kbmMW_Rest('method:get, path:"addcontact/{name}"')]
  function AddContact([kbmMW_Rest('value:"{name}"')] const AName:string):string; overload;

  [kbmMW_Auth('role:[READWRITER], grant:true')]
  [kbmMW_Rest('method:delete, path:"contact/{id}"')]
  function DeleteContact([kbmMW_Rest('value:"{id}"')] const AID:string):boolean;
 end;

 

What happens behind the scenes is that kbmMW automatically define resource names for the functions like this: MyREST..AddContect, MyREST..GetContacts etc.

Notice the extra dot! If we had defined the service to have a version, when we created it, that would be put between the dots.

As you can see, the resource name is just a string, and you can define all the resources you want to yourself, but know that if you use kbmMW smart services, it will automatically define resource names in the above format.

kbmMW will also automatically ask the authorization manager to validate that it is allowed to use a resource, upon a call from any client.

You can choose to make finer grained authorization by manually calling the authorization manager for validation of a call like this:

var
  res:TkbmMWAuthorizationStatus;
  sMessage:string;
begin
...
  res:=AuthorizationManager1.IsAuthorized(logintoken, 'YOURRESOURCENAME', sMessage);

res can have the value of mwasAuthorized, mwasNotAuthorized or mwasConstrained.

mwasConstained means that the authorization might be given under different circumstances (different time on day or similar). The returned sMessage may explain in more detail what was the reason that the access was denied.

In a kbmMW smart service, you can get the login token (logintoken) as an argument to the method like this:

 [kbmMW_Auth('role:[READER], grant:true')]
 [kbmMW_Rest('method:get, path:"someCall"')]
 function SomeCall([kbmMW_Arg(mwatToken)] const AToken:string):boolean;

When the SomeCall method is called, its AToken argument contains the logintoken.

You can also access the services ClientIdentity.Token property instead from within your methods if you do not want the token to be part of the argument list of your method call.

Now your REST server is protected by SSL and calls to it’s functionality limited by login.

There are many more features in the authorization manager, which I have not explained here, but visit our site at http://www.components4developers.com, and look for the kbmMW documentations section for whitepapers.

If you like this, please share the word about kbmMW wherever you can and feel free to link, like, share and copy the posts of this blog to where you find they could be useful.

 

 

REST easy with kbmMW #1

REST servers are very easy to make using kbmMW.

First we make a server application (or service… it’s up to you).

In this case we will add a simple form both providing a GUI and a place for our kbmMW components.

In Delphi click File – New – VCL Forms application

Add one of each of the following kbmMW components to the form:

  • TkbmMWServer
  • TkbmMWTCPIPIndyServerTransport

2017-05-20 14_50_54-Project6 - RAD Studio 10.1 Berlin - Unit7.png

Set the Server property of kbmMWTCPIPIndyServerTransport1 to kbmMWServer1.

Double click the Bindings property of kbmMWTCPIPIndyServerTransport1 to open its editor. Add a binding for 0.0.0.0 port 80, which is the default HTTP server port. You can choose any other binding you want, but make sure to tell your REST users which port to access.2017-05-20 14_53_34-Project6 - RAD Studio 10.1 Berlin - Unit7.png

Set the Streamformat property of kbmMWTCPIPIndyTransport1 to REST.2017-05-20 14_56_15-Project6 - RAD Studio 10.1 Berlin - Unit7.png

Save your project. Saving it will prompt Delphi to update your uses section with required units.

Double click on the form, to create a form OnCreate event handler.

Enter two lines of code so it looks like this:

procedure TForm7.FormCreate(Sender: TObject);
begin
 kbmMWServer1.AutoRegisterServices;
 kbmMWServer1.Active:=true;
end;

Locate the interface section’s uses clause and add an additional unit kbmMWRESTTransStream.

Save again.

Now your unit code should look similar to this:

unit Unit7;

interface

uses
 Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
 Vcl.Controls, Vcl.Forms, Vcl.Dialogs, kbmMWCustomTransport, kbmMWServer,
 kbmMWTCPIPIndyServerTransport, kbmMWRESTTransStream;

type
 TForm7 = class(TForm)
 kbmMWServer1: TkbmMWServer;
 kbmMWTCPIPIndyServerTransport1: TkbmMWTCPIPIndyServerTransport;
 procedure FormCreate(Sender: TObject);
 private
 { Private declarations }
 public
 { Public declarations }
 end;

var
 Form7: TForm7;

implementation

{$R *.dfm}

procedure TForm7.FormCreate(Sender: TObject);
begin
 kbmMWServer1.AutoRegisterServices;
 kbmMWServer1.Active:=true;
end;

end.

Basically we now have the foundation for a REST capable web server.

Let’s add some functionality that can be called from any REST client.

In Delphi, Click FileNewOtherComponents4Developers Wizards and select the kbmMW Service Wizard. Click OK.

2017-05-20 15_09_18-Project6 - RAD Studio 10.1 Berlin - Unit7.png

Before continuing to select the type of kbmMW service we will add, we need to decide what type of REST server we want to create. It can be a pure REST server, which only serves data from your own code, or it can be a regular web server, which will also be able to serve data from files on disk, like html templates, images, CSS files etc.

For our purpose we just want to make a pure REST server, so we select
Smart service/kbmMW_1.0.

If you wanted it to be able to serve files from disk, and even proxy requests on to other FastCGI compatible servers (like PHP etc) you would have chosen HTTP Smart service.

2017-05-20 15_13_56-kbmMW service wizard.png

Click the funny looking next button. Type in the default name your REST service should be known as. In this sample, I’ve called it MyREST.

2017-05-20 15_15_16-Project6 - RAD Studio 10.1 Berlin - Unit7.png

Click next until you get to this page, then click the green checkmark button

2017-05-20 15_16_42-kbmMW service wizard.png

Now an almost empty service has been generated for you.

On the surface it looks like a regular TDataModule, and as such can contain any component that you can put on a TDataModule. But right now we are more interested in its code. Press F12 to switch to code view.

Browse past the explanatory remarks at the top, until you get to the actual code.

type

[kbmMW_Service('name:MyREST, flags:[listed]')]
[kbmMW_Rest('path:/MyREST')]
 // Access to the service can be limited using the [kbmMW_Auth..] attribute.
 // [kbmMW_Auth('role:[SomeRole,SomeOtherRole], grant:true')]

TkbmMWCustomSmartService8 = class(TkbmMWCustomSmartService)
 private
 { Private declarations }
 protected
 { Protected declarations }
 public
 { Public declarations }
 // HelloWorld function callable from both a regular client,
 // due to the optional [kbmMW_Method] attribute,
 // and from a REST client due to the optional [kbmMW_Rest] attribute.
 // The access path to the function from a REST client (like a browser)+
 // is in this case relative to the services path.
 // In this example: http://.../MyREST/helloworld
 // Access to the function can be limited using the [kbmMW_Auth..] attribute.
 // [kbmMW_Auth('role:[SomeRole,SomeOtherRole], grant:true')]
 [kbmMW_Rest('method:get, path:helloworld')]
 [kbmMW_Method]
 function HelloWorld:string;
 end;

implementation

uses kbmMWExceptions;

{$R *.dfm}


// Service definitions.
//---------------------

function TkbmMWCustomSmartService8.HelloWorld:string;
begin
 Result:='Hello world';
end;

The interesting bits are shown above in bold.

If you compile and run your application now, you have a REST only capable webserver which have one function… HelloWorld taking no arguments, and that returns a string.

Open up your favorite web browser and lets test the function by typing this in the address field:

http://localhost/MyREST/helloworld

Make sure that case is correct, since the HTTP standard describes that the URL part of the address must be case sensitive. If you would write http://localhost/MyREST/HelloWorld instead you would be told that the request is invalid.

This is all nice… but my REST client expect to receive a JSON object, not just simple text.

Ok.. I’ll show 3 ways to do that… the very manual way, the semi automated way and the fully automated way.

The manual way. Change the HelloWorld function to look like this:

function TkbmMWCustomSmartService8.HelloWorld:string;
begin
 Result:='{''result'':''Hello world''}';
end;

The REST client will receive an anonymous object with a property called result, containing “Hello world”.

The semi automated way:

uses kbmMWExceptions
 ,kbmMWObjectNotation
 ,kbmMWJSON;

{$R *.dfm}


// Service definitions.
//---------------------

function TkbmMWCustomSmartService8.HelloWorld:string;
var
 o:TkbmMWONObject;
 jsonstreamer:TkbmMWJSONStreamer;
begin
 o:=TkbmMWONObject.Create;
 jsonstreamer:=TkbmMWJSONStreamer.Create;

 o.AsString['result']:='Hello world';
 Result:=jsonstreamer.SaveToUTF16String(o);

 jsonstreamer.Free;
 o.Free;
end;

This allows you to create complex JSON documents pretty easily. The cool part is that since we use kbmMW’s object notation framework, we could have chosen to stream it as XML or YAML or BSON or MessagePack instead by simply instantiating the appropriate streamer.

The automated way:

type

TMyResult = class
private
 FResult:string;
public
 property Result:string read FResult write FResult;
end;

 [kbmMW_Service('name:MyREST, flags:[listed]')]
 [kbmMW_Rest('path:/MyREST')]
 // Access to the service can be limited using the [kbmMW_Auth..] attribute.
 // [kbmMW_Auth('role:[SomeRole,SomeOtherRole], grant:true')]

TkbmMWCustomSmartService8 = class(TkbmMWCustomSmartService)
 private
 [kbmMW_Rest('method:get, path:helloworld, anonymousResult:true')]
 [kbmMW_Method]
 function HelloWorld:TMyResult;
 end;

implementation

uses kbmMWExceptions;

{$R *.dfm}

// Service definitions.
//---------------------

function TkbmMWCustomSmartService8.HelloWorld:TMyResult;
begin
 Result:=TMyResult.Create;
 Result.Result:='Hello world';
end;

initialization
 TkbmMWRTTI.EnableRTTI(TkbmMWCustomSmartService8);
 kbmMWRegisterKnownClasses([TMyResult]);
end.

The automated way simply means returning an object with the desired information. kbmMW will automatically convert the object to JSON (because we are using the REST streamformat).

To make sure that kbmMW knows about the object type, we register it via the kbmMWRegisterKnownClasses. If we didn’t, kbmMW would complain that it do not know about the object.

Do not worry about the TMyResult instance being leaked. kbmMW will automatically free it when it goes out of scope. If you specifically do not want the returned object to be freed by kbmMW, you can tell so by including freeResult:false in the kbmMW_Rest attribute for the HelloWorld method.

Also notice that the kbmMW_Rest attribute now includes anonymousResult:true.

This tells kbmMW that we want the resulting JSON to be anonymous. If we didn’t include that attribute setting, the result would have looked like this:

{"TMyResult":{"Result":"Hello world"}}

Which is not necessarily wrong, but different.

There are lots of control options of how the object should be streamed by setting various attributes on it. One can for example choose that the Result property should be returned under a different name etc.

kbmMW also understands returning TkbmMemTable instances, arrays and many other types of information, so it is really easy to REST’ify your kbmMW business functionality with almost no lines of additional code.

As a final comment, since the HelloWorld method is also tagged with the attribute [kbmMW_Method], it is also callable by native kbmMW clients.

Feel free to share, comment and like this post 🙂

 

kbmMW Scheduler Tidbits #2

Next version of kbmMW will continue to improve on the already very powerful job scheduler built into the framework.

Now UNIX CronTab like scheduling can be made. Examples:

// At the 8 and 9 minutes past the hour
 Scheduler.Schedule(OnTestEvent).Cron('8,9 * * * * *').NamedAs('cron1').Activate;

// At the exact minute of the hour when this event is scheduled
 Scheduler.Schedule(OnTestEvent).Cron('? * * * * *').NamedAs('cron2').Activate;

// Every 5 minutes
 Scheduler.Schedule(OnTestEvent).Cron('*/5 * * * * *').NamedAs('cron3').Activate;

// Every minute at Monday between 8:20 and 8:30
 Scheduler.Schedule(OnTestEvent).Cron('20-30 8 * * Mon *').NamedAs('cron4').Activate;

// Every second minute at Monday between 8:20 and 8:30
 Scheduler.Schedule(OnTestEvent).Cron('20-30/2 8 * * 1 *').NamedAs('cron5').Activate;

// Every second minute at Monday and Tuesday between 8:20 and 8:30
 Scheduler.Schedule(OnTestEvent).Cron('20-30/2 8 * * 1,2 *').NamedAs('cron6').Activate;

// Every second minute at Monday and Tuesday, in March between 8:20 and 8:30
 Scheduler.Schedule(OnTestEvent).Cron('20-30/2 8 * mar 1,2 *').NamedAs('cron7').Activate;

kbmMW supports an extended CronTab format which includes these features.

  • Optional year
  • List of values
  • Ranges
  • Intervals
  • Optional textual day of week
  • Optional textual month name
  • Start time indicator

Syntax for cron: minute hour day month dayofweek year
Year is optional.
Thus 5 or 6 space separated values must be given.

Each of the values can be:

* indicating any value.
? indicating value at schedule time
1,2,3.. list of values
1,4-5,7… list of values
1-5 range of values
*/2 interval… every 2
3-15/3 interval… every 3 starting at 3, ending at 15
Interval values can be used for minute, hour, day, month and year values.

Month can be given as a value 1..12 or text jan, feb, mar, apr,  may, jun, jul, aug, sep, oct, nov, dec.

DayOfWeek can be given as value 1 (monday)..7 (sunday) or mon, tue, wed, thu, fri, sat, sun.

All the other fluent expression supported by kbmMW’s scheduler, like InitialDelay, Starting, Ending, Occurance etc. can be added after the Cron statement.

In addition to Cron, a number of fluent methods like AtYears, AtMonths, AtDays, AtHours, AtMinutes, AtSeconds has been added to allow setting up Cron like schedules without using the Cron string format.

Digging for wORMs

kbmMW ORM and SQL rewriting explained in beautiful combination with kbmMW Smart services and kbmMW Smart clients.

I’ve promised to release v5 with a new very cool feature (in beta), which I’ll explain a bit about now: kbmMW ORM (object relational model).

Whats an ORM? It can be interpreted in multiple ways. In our way we interpret it as a way to persist, query and manipulate objects to and from a data storage of some sort.

We have focused more on the object persistence part of the ORM, than on the automatic relational management of master/detail scenarios. There are good and bad things about fully automatic relational cascade type storage and data manipulation. In some cases the bad things outweighs the good, and sometimes visa versa.

Show me the code!

Well.. its simple… create an object, tell the framework to persist the object and you have stored your data. It sounds too easy doesn’t it? It really _is_ easy.

This is an ordinary class, for a TPerson. This example contains some funny types, like kbmMWNullable, but thats just a convenience that makes some things even easier. However you could choose to use  plain simple regular native strings and integer and other native types.

type

 [kbmMW_Table('name:person, index:{name:i1,field:name,descending:false}, index:{name:i2,unique:true,fields:[{name:name,descending:true},{name:age}]')]
 TPerson = class
 public
  FID:kbmMWNullable;
  FName:kbmMWNullable;
  FAddress:kbmMWNullable;
  FAge:kbmMWNullable;

 published
  [kbmMW_Field('name:id, primary:true, generator:shortGuid',ftString,40)]
  property ID:kbmMWNullable read FID write FID;

  [kbmMW_Field('name:name',ftWideString,50)]
  property FullName:kbmMWNullable read FName write FName;

  [kbmMW_Field('name:address',ftWideString,50)]
  property Address:kbmMWNullable read FAddress write FAddress;

  [kbmMW_Field('name:age',ftInteger)]
  property Age:kbmMWNullable read FAge write FAge;
end;

initialization

 kbmMWRegisterKnownClasses([TPerson,TObjectList]);

end.

What you may also notice are some custom attributes named kbmMW_Table and kbmMW_Field.

kbmMW_Table tells the kbmMW framework something about how objects of this type are to be stored in data storage. It includes the desired table name and optional additional indexes that would be nice to have of performance reasons.

kbmMW_Field tells the framework that the following property or field is to be persisted, and under what data storage field name it will be persisted. Further it describes how that data storage field should be, type, size, precision and more.

The ID property has more information attached to it. The framework is being told that this field should be the primary field (a primary field should always be declared), and that the fields value is automatically generated upon storage in the database. In this case the value will contain a so called short GUID, which is a shortened version of a regular GUID.

The field refers to a generator. kbmMW ORM supports a number of generators:

  • GUID – Returns a GUID formatted as a regular GUID
    {123e4567-e89b-12d3-a456-426655440000}
  • SHORTGUID – Returns a short GUID where braces and dashes are missing:
    123e4567e89b12d3a456426655440000
  • SEQUENCE – Returns next unique number from a sequence. Provide name of sequencer in sequence property and optional sequencestart property (not supported by all databases!)
  • DATETIME – Returns a date time value, formatted according to the dateFormat property

In addition the field type can be set to ftAutoinc, just like any regular TDataset based databases. In this case the database will generate a unique number, which the framework will pickup automatically and return into the persisted object.

So how do we get around to actually persist anything?

First we create an ORM instance, linking it to a connection pool. The connection pool is responsible for the actual connection to the data storage.

 orm:TkbmMWORM;
...
 orm:=TkbmMWORM.Create(kbmMWSQLiteConnectionPool1);

In this case we have chosen a kbmMWSQLite connection pool, but most databases and connection pools are supported. I will write a bit more about that later.

The connection pool and a metadata component matching it (kbmMW will automatically attempt to find the right one if none has been setup) was put on the form at design time.

2017-04-17 18_47_59-SettingsOne single thing you need to setup on the connection pool, is the database name, and if you want the system to automatically create the SQLite database if none exists, then also set its SQLiteOptions property to contain mwsloAutoCreateDB, and set its MetaData to point on the kbmMWSQLiteMetaData1 component.

The metadata component is actually not mandatory to place on the form, and is only shown for completeness. kbmMW will automatically generate a metadata component instance that match the used connection pool, if none has been provided.

As you may be able to see from the screendump (if you have been using kbmMW before), is that this form is part of a simple n-tier server ORM sample which is included with v5.

Lets look at the code again… we now want the ORM to automatically create all relevant tables in the data storage. Im talking about table’s here, because that’s what most people use for storing data, but it could also be other types of storage, which have other concepts for storage than tables. As long as a connection pool, with accompanying metadata exists for that storage, it will work fine with kbmMW ORM.

Having the table created is as simple as this

orm.CreateTable(TPerson);

CreateTable can take an array of classes, if multiple tables should be created in one go.

You can test if a table (or tables) already exists using

 if not orm.ExistsTable(TPerson) then
....

CreateTable will only create tables if they are missing, but using ExistsTable may make it easier for you, to know when you can prepopulate newly created table with whatever bootstrap information you like to put into them from start.

You can use DeleteTable(TPerson) to delete the table and its data, and you can use PurgeTable to leave the table, but remove its contents.

How do we put some data into the table?

We do that by instantiating instances of TPerson like this

 p:=TPerson.Create;
 p.FullName:='Hans Hansen';
 p.Address:='Hans Home Address 1';
 p.Age:=65;
 orm.Persist(p);

After Persist has run successfully, you will find that p.ID has a value, even though we didn’t put one originally. If an object do not have a value for the field that was marked as primary, then that object is understood as a new object.

Thus in the future (for as long as we keep the instance pointed to by p) alive, kbmMW ORM will know that future persisting of it means that the existing record in the table will be updated rather than a new added.

To add more records, simply create more instances and persist them.

Remember that you own the TPerson instances that you create, and can do with them what you want, which also means that you are responsible for freeing them again when you no longer need them.

Freeing the instance will not affect what has been persisted.

If you make changes in the object and want those changes persisted, simply call orm.Persist(p) on the object again.

If you want to delete an object from persistence, you do

orm.Delete(p);

You still have the p instance in memory, but it is now gone from storage.

If you have a collection or an array of TPeople you can Persist and Delete those the same way as already shown.

Persisting data is no fun, unless you can retrieve them again at some point.

kbmMW ORM supports a vast number of ways to search and retrieve data via the various Query methods.

Lets say you want to retrieve an object with a specific primary key

var
 p:TPerson;
...
 p:=orm.Query<TPerson>('123e4567e89b12d3a456426655440000');

Now p will either be nil or point to a matching instance of TPerson. You own this object instance and may free it when you want.

Or if you know there is only 0 or 1 record in the table, you can simply do

p:=orm.Query<TPerson>;

Even if there are more records, only the first one will be returned to you. It should obviously be used with common sense because it does request all records from the data storage, and they may come in any order.

If you want a complete list of all persons known by the data storage

var
 p:TObjectList<TPerson>;
...
 p:=orm.QueryList<TPerson>;

Now… this is all nice and easy.. but what if I want to search for all people with a specific age?

var
 p:TObjectList<TPerson>;
...
 p:=orm.QueryList<TPerson>(['age'],[65]);

Now only people of the age 65 is returned.

var
 p:TObjectList<TPerson>;
...
 p:=orm.QueryList<TPerson>(['age'],[65],mwoqoGT);

Now only people older than 65 is returned. If you only wanted one of them you could have used Query<TPerson>… instead of QueryList.

You can specify multiple fields and values in your search this way, but all are controlled by the given operator (default mwoqoEQ = equal).

If you just wanted a count

var
 i:integer;
...
 i:=orm.QueryCount<TPerson>(['age'],[65],mwoqoGT);

You can use various aggregates, like QueryCount, QuerySum, QueryAvg, QueryMin, QueryMax and QueryStdDev or the generic QueryAggregate method.

Or if you really want the data as a TDataset instead, you can do

var
 ds:TDataset;
...
 ds:=orm.QueryDataset<TPerson>(['age'],[65],mwoqoGT);

This will return a dataset to, which you can use as any other dataset. The dataset is however not directly connected with the data storage behind, so changes in the dataset needs to be either resolved using kbmMW’s regular resolving mechanism (which is nice, but not ORM), or by converting the dataset back to a set of objects you can persist.

The field name given is the name your property is known as. So if you would have searched on a name, you would have specified ‘FullName’ as the field name, rather than ‘name’ as it’s given in the data storage.

With kbmMW ORM you must generally always refer to the object’s property names. There is an exception to that, which is shown further down.

For example if you want to get all deleted persons from the dataset, to let the ORM actually delete those:

var
 p:TObjectList<TPerson>;
...
 p:=orm.ListFromDataset<TPerson>(ds,[usDeleted]);
 orm.Delete(p);
 o.Free;

If you want to know if data has changed (been inserted or modified) in an object, you can

if orm.HasChanged(p) then
...

This is here where kbmMWNullable and TkbmMWDateTime comes into play. Regular native types like string, integer etc. can’t hold the value Null, and they also can’t hold any information about if their value has been changed.

But if you wrap the types in kbmMWNullable, or use TkbmMWDateTime instead of TDateTime, you suddenly have those abilities which the framework can take advantage of.

Looking at the FullName property in our TPerson object, you can see that it was wrapped in kbmMWNullable. That means we at any time can request information about if its null, or modified like this

if p.FullName.IsNull then
...
if p.FullName.Modified then
...

And we can clear its modified flag anytime if we need to

p.FullName.Modified:=False;

And we can set its value to null

p.FullName.Clear;

By default, all properties are marked as non modified when they have been returned as a result of one of the Query methods.

This way, Persist and other UpdateIfChanged / InsertIfChanged methods, will only resolve data back to the data storage if any changes have been made to the data in the object. If you are using regular non wrapped properties, then kbmMW ORM can only assume that the data may have changed, and it will attempt to update all provided objects regardless.

Most of the time, you really want to make more complex searches when querying for data. We have chosen to build in a uniform SQL based query language into kbmMW ORM which is not only easy to understand for most, but is automatically translated on the fly, to the query language used by the data storage.

var
 o:TObjectList<TPerson>;
...
 o:=orm.QueryList<TPerson>('SELECT * FROM uData.TPerson WHERE age>65 ORDER BY FullName'));

Notice that you select from uData.TPerson. That is a so called qualified class name, which means that there is no doubt about which exact TPerson you want to search on. You can have multiple TPerson classes defined in multiple units and that’s perfectly legal. While querying on them, just remember always to prefix the unit name.

Also notice the ORDER BY part. It refers to FullName which is the property in the TPerson class. The query parser also supports joins. But remember that only the fields that match the requested object type will be returned.

var
 o:TPersonAccount;
...
 o:=orm.Query<TPersonAccount>('SELECT p.id, name, value FROM uData.TPerson p, uData.TAccount a WHERE a.PID=p.ID');

To have all data returned, you will have to have define a TPersonAccount class somewhere which contains minimum the fields ID, Name and Value. Excess fields in the query will be silently ignored.

[kbmMW_VirtualTable]
 TPersonAccount = class
 private
   FID:string;
   FName:kbmMWNullable<string>;
   FValue:kbmMWNullable<double>;
 public
   [kbmMW_Field('name:id')]
   property ID:string read FID write FID;

  [kbmMW_Field('name:name')]
  property Name:kbmMWNullable<string> read FName write FName;

  [kbmMW_Field('name:value')]
  property Value:kbmMWNullable<double> read FValue write FValue;
 end;

Notice that this class has been given the attribute kbmMW_VirtualTable. A virtual table is a table that do not exist in the data storage, and as such if you try to CreateTable(TPersonAccount) an exception will be thrown telling you that its a virtual table that is only meant for transient uses.

Also notice that in this case we have defined ID as a regular native string instead of wrapping it with kbmMWNullable. As we will always have a value in the ID property, we can safely choose not to use kbmMWNullable, but its perfectly fine to wrap even this one with kbmMWNullable to follow a simple rule of thumb and not having to judge for each property type.

The kbmMW ORM SQL syntax is fairly complete, and sticking to it allow you to easily move your application between the different database types supported by kbmMW, in particular SQLite, Oracle, MSSQL, PostgreSQL, MySQL and Interbase/Firebird. Also many other databases are supported. If a database specific metadata component (which takes care of the rewriting of SQL) do not exist, then kbmMW ORM will attempt to fall back to a generic SQL metadata component, which will work in many cases.

However if there is a special query you want to use, which kbmMW ORM SQL do not directly support, you can send it raw, provided your orm instance will allow it.

To allow for custom native query statements, make sure that
orm.QueryMode:=mwoqmNative or mwoqmMixed.

It is default mwoqmMW.

Setting it to native, prevents kbmMW ORM to rewrite anything, and all query statements are considered native.

Setting it to mixed, will make kbmMW ORM assume that the query statements are kbmMW ORM SQL, but allow for you to put a # in front of a statement to send a native statement in a query.

var
 o:TObjectList<TPerson>;
...
 o:=orm.QueryList(TPerson,'#SELECT * FROM Person');

Notice that the table name given, must match the table name in the data storage.

But what about n-tier scenarios?

I started out with telling about that this article was based on an n-tier ORM sample, so it makes sense to show the remaining parts that enables a client to query for people, update changes to them and delete them.

kbmMW v5 was from outset designed to be the easiest to use n-tier product ever. With it came the new smart service and smart clients, and what else makes sense than to use those along with the ORM. This makes it possible to create both a server and a client, requiring only a few lines of business code almost totally eliminating boiler plate code (ie. code that does nothing for your business logic, but is required to glue things together).

Thus we will create a smart service with 3 methods, StorePersons, DeletePersons and GetPersons. As we have put the orm instance on the main form (Form1) we refer to that from here.

[kbmMW_Service('name:ORMDEMO')]
 TkbmMWCustomService2 = class(TkbmMWCustomSmartService)
 public
  // This lists stores changes (but not removed items) in the list of persons.
  [kbmMW_Method]
  procedure StorePersons(const APersons:TObjectList<TPerson>);

  // This lists deletes persons in the list.
  [kbmMW_Method]
  procedure DeletePersons(const APersons:TObjectList<TPerson>);

  // This method returns a list of persons.
  [kbmMW_Method]
  function GetPersons:TObjectList<TPerson>;
 end;

implementation

uses
 Unit1;

{$R *.dfm}

procedure TkbmMWCustomService2.StorePersons(const APersons:TObjectList<TPerson>);
begin
 // The APersons list is owned by this function, and must be manually freed.
 Form1.orm.Persist(APersons);
 APersons.Free;
end;

procedure TkbmMWCustomService2.DeletePersons(const APersons:TObjectList<TPerson>);
begin
 // The APersons list is owned by this function, and must be manually freed.
 Form1.orm.Delete(APersons);
 APersons.Free;
end;

function TkbmMWCustomService2.GetPersons:TObjectList<TPerson>;
begin
 // Return all persons (people).
 Result:=Form1.orm.QueryList<TPerson>;
end;

initialization
 TkbmMWRTTI.EnableRTTI(TkbmMWCustomService2);

This is really really simple isn’t it?  Only business logic code and nothing else.

In the FormCreate event handler of the main form Form1, where we have also put the orm creation, we add

 // Register all services automatically.
 // Services will only be autoregistered if they have a kbmMW_Service attribute.
 server.AutoRegisterServices;

In the event handler of the Listen button on Form1 we write

 server.Active:=not server.Active;

That’s all!

Now to the client.

2017-04-17 20_52_08-Project1 - RAD Studio 10.1 Berlin - Unit1

The client has a connect button, a button requesting a list of TPerson and a button to store changes made to TPerson data.

In addition it contains a string grid and a mtPersons kbmMemTable which are live bound together along with the navigator.

Embarcadero’s live binding is in some ways nice, but it is not terribly flexible, so it will only bind to native object properties and fields. In our case we have kbmMWNullable wrapped properties, which are simple to use in code, but which are not understood by livebinding.

This can be circumvented in a number of ways. One is to add a number of alternative properties to the TPerson object, which exposes the same data, but as native types.

This could look like this

[kbmMW_Table('name:person, index:{name:i1,field:name,descending:false}, index:{name:i2,unique:true,fields:[{name:name,descending:true},{name:age}]')]
 TPerson = class
 private
  FID:kbmMWNullable<string>;
  FName:kbmMWNullable<string>;
  FAddress:kbmMWNullable<string>;
  FAge:kbmMWNullable<integer>;

  function GetID:string;
  procedure SetID(AValue:string);
  function GetName:string;
  procedure SetName(AValue:string);
  function GetAddress:string;
  procedure SetAddress(AValue:string);
  function GetAge:integer;
  procedure SetAge(AValue:integer);
 published
  [kbmMW_Field('name:id, primary:true, generator:shortGuid',ftString,40)]
  property ID:kbmMWNullable<string> read FID write FID;

  [kbmMW_Field('name:name',ftWideString,50)]
  property FullName:kbmMWNullable<string> read FName write FName;

  [kbmMW_Field('name:address',ftWideString,50)]
  property Address:kbmMWNullable<string> read FAddress write FAddress;

  [kbmMW_Field('name:age',ftInteger)]
  property Age:kbmMWNullable<integer> read FAge write FAge;

  // One way of livebinding to kbmMWNullable or kbmMWDateTime fields is
  // to add an alternative access to them.
  // LiveBinding only supports basic data types.
  // Remember to kbmMW_Ignore the properties to tell
  // kbmMW not to attempt to stream them.
  // As kbmMWNullable and TkbmMWDateTime values retain
  // information about nullability and modification,
  // it may be preferred to use these types over
  // regular string, int, TDateTime etc. types.
  // If the number of properties of such types is large,
  // writing many alternative setters and getters may become
  // tedious. Instead you can (as this sample shows in unit1.pas), convert
  // the objects to a dataset (a proxy) which is easy to databind with.

  [kbmMW_Ignore]
  property LBID:string read GetID write SetID;

  [kbmMW_Ignore]
  property LBName:string read GetName write SetName;

  [kbmMW_Ignore]
  property LBAddress:string read GetAddress write SetAddress;

  [kbmMW_Ignore]
  property LBAge:integer read GetAge write SetAge;
 end;

Notice that the extra properties are not wrappe with kbmMWNullable. They have also been marked with the attribute kbmMW_Ignore, to tell the kbmMW framework that they are not to be marshalled (serialized).

Their only purpose is to allow for easy object live binding and is basically boiler plate code that we want to avoid.

There is another way that require less typing, and converting the objects to a dataset and back again when needed.

kbmMW ORM provides easy support for that.

In this case, we have already put a mtPerson kbmMemTable on the form at designtime, to make it easy to do designtime live binding.

All we need is to populate it with some data. This happens in the Get persons buttons eventhandler.

But first we need to prepare the smart client.

This can happen in the Connect buttons event handler.

// A form field
c:IkbmMWSmartClient;
...
// Connect button's event handler
c:=TkbmMWSmartRemoteClientFactory.GetClient(Transport,'ORMDEMO');

Then let us make a few methods to get and send data.

function TForm1.GetPersons:TObjectList<TPerson>;
begin
 // Request a list of persons.
 Result:=Use.AsObject<TObjectList<TPerson>>(c.Service.GetPersons);
end;

procedure TForm1.StorePersons(APersons:TObjectList<TPerson>);
begin
 // Store a list of persons.
 c.Service.StorePersons(Use.Arg(APersons));
end;

procedure TForm1.DeletePersons(APersons:TObjectList<TPerson>);
begin
 // Store a list of persons.
 c.Service.DeletePersons(Use.Arg(APersons));
end;

And in the Get Persons buttons event handler put

orm.ToDataset<TPerson>(mtPersons,GetPersons,true);

Now the grid is populated with data, and you can work the data in any way you want. As the data is available in a kbmMemTable, sorting, searching , filtering and other things are very easy to do.

At some point, the data may have been modified, and it should be sent to the server for updating.

In the Store Persons buttons event handler put

var
 persons:TObjectList<TPerson>;
begin
 persons:=orm.ListFromDataset<TPerson>(mtPersons,[usDeleted]);
 if persons.Count>0 then
   DeletePersons(persons)
 else
   persons.Free;
 persons:=orm.ListFromDataset<TPerson>(mtPersons);
 StorePersons(persons);
end;

This first sends a list of deleted TPerson’s to the server’s DeletePersons method (if any were deleted), and then a list of remaining data. In fact that list could also be limited to only inserted or modified data by adding the argument [usModified,usInserted] to the ListFromDataset method call. But right now we just ship all what is visible.

What happens on the server, is that it will know which fields was changed and only update the records that are relevant . Remember this require that all fields in the TPerson object are either wrapped with kbmMWNullable or are of type TkbmMWDateTime.

2017-04-17 21_16_09-Project1 - RAD Studio 10.1 Berlin - Unit1 [Running] [Built]

The combination of smart services, smart clients, advanced object marshalling, and the new ORM really makes it extremely easy to build multi tier applications in a true RAD way!

Happy wORMing!