Teaser: kbmMW safety first… Random numbers and great passwords!

Usually Delphi’s built in 32 bit random generator is sufficient for most tasks, like generating random numbers for some tests, or a game or something similar. However the random generator is, if one focus on security, not strong enough to be used for cryptographic uses, like password generation.

Random numbers

In next version of kbmMW, a set of random number generators has been added for generating 32bit and 64bit random values in addition to supporting the standard Delphi Randon method.

32 bit random generators in kbmMW:

  • TkbmMWRandomDelphi
  • TkbmMWRandomPCGUInt32
  • TkbmMWRandomMersenneTwisterUInt32

64 bit random generators in kbmMW:

  • TkbmMWRandomSplitMix
  • TkbmMWRandomXoroshiro128Plus
  • TkbmMWRandomXoroshiro1024
  • TkbmMWRandomMersenneTwisterUInt64

They all follow the same principle, and it’s easy to replace one with another, or even add another custom random generator.

To see the randomness of the generators, one can make a lot of random X and Y values and plot them in a fine masked grid. A perfectly random function (in reality that does not exist in current computers) ought to spread the numbers fairly evenly across the grid.

2017-02-01-01_46_40-xbox
The 32 bit standard Delphi random plot

2017-02-01-01_48_03-xbox
The 32 bit PCG random plot

2017-02-01-01_49_17-xbox
The 32 bit Mersenne Twister random plot

2017-02-01-01_50_42-xbox
The 64 bit split mix random plot

2017-02-01-01_52_53-xbox
The 64 bit Xoroshiro 128+ random plot

2017-02-01-02_14_37-xbox
The 64 bit Xoroshiro1024 random plot

2017-02-01-02_17_03-xbox
The 64 bit Mersenne Twister random plot

As can be seen, the Delphi random generator is pretty bad in producing values across the entire value space, while the others show a much more even spread as would be expected by a fair random generator.

Choosing a random number generator for a particular purpose can be difficult. One has to weigh in speed vs randomness vs full cycle time and other factors. Basically the perfect randon number generator do not exist at this time for regular computers.

These pages explains a bit about their weaknesses and strengths:
http://xoroshiro.di.unimi.it/
http://www.pcg-random.org/

But at least now you have a better choice. And in fact this brings us to the next section. Generation of strong passwords.

Strong passwords

Most software today supports some sort of user login, where a password is required. kbmMW certainly supports that, through for example the authorization manager.

But the challenge is to force users to use some good passwords, which they still have a chance to remember without writing down.

As a first for Delphi, kbmMW now also supports multiple password generator algorithms to encourage use of strong passwords.

kbmMW comes with two password generators and a framework on which new custom password generators can be built:

  • TkbmMWMixerPasswordGen
  • TkbmMWKoremutakePasswordGen

The Mixer password generator supports selecting the minimum and maximum number of digits, minimum and maximum number of punctuation characters, unicase or mixed case, and minimum length of a generated password.

The Koremutake password generator generates a random 64 bit value, and converts that 64 bit value into a string consisting of two and three character character groups, forming a semi pronounceable password.

Using the password generators is very simple:

var
   pg:TkbmMWMixerPasswordGen;
begin
     pg:=TkbmMWMixerPasswordGen.Create;
     try
        // Optionally set length, digits, punctuation and case settings.
        // Default a password will be minimum 8 characters long,
        // contain from 1 to 4 digits, no punctuation characters
        // and use mixed case alpha characters.
        Memo1.Lines.Add(pg.Generate);
     finally
        pg.Free;
     end;
end;

This could output: 69vcRPhw

var
   pg:TkbmMWKoremutakePasswordGen;
begin
     pg:=TkbmMWKoremutakePasswordGen.Create;
     try
        Memo1.Lines.Add(pg.Generate);
     finally
        pg.Free;
     end;
end;

This could output: GUMIPAVYGRYTIFOFYSI

We will continue to monitor the various options for password generators, and provide support for them as we find them interesting.

Teaser : kbmMW’s new object notation

kbmMW now supports XML, JSON, BSON, Messagepack and YAML natively, and includes a new common Object Notation format for easy conversion.

JSON and XML has been supported natively by kbmMW for years. Later kbmMW also supported marshalling of native Delphi objects to and from JSON and XML.

In addition to it being a very nice way to stream Delphi objects, the feature also allowed for a way to convert XML to JSON and visa versa.

However, the upcoming version of kbmMW now includes a new object notation framework, that makes it easy to convert from one notation to another, without going through a native Delphi object.

The new version of kbmMW not only supports JSON and XML, but now also YAML (which is a more human readable format) and the compact binary Messagepack and BSON formats, natively, both for creating your own object notation trees in code, and for marshalling Delphi objects to and from.

With the many new object notation formats supported by kbmMW, it also made sense to find a way to make it easy to convert between them, without having to marshal to and from native Delphi objects.

For that reason the kbmMW Object Notation format was invented. The object notation format is based on a number of classes:

  • TkbmMWONCustomObject (from which the classes listed below descends)
  • TkbmMWONObject (for key/value type data)
  • TkbmMWONArray (for array type data)
  • TkbmMWONNative (for string, integer, floats, date/time, binary values, custom native values and more)

In addition a number of abstract stream classes was developed, which purpose is to handle converting the object notation trees to and from various string or binary formats, and a set of abstract classes that handles marshalling object notation trees to and from native Delphi objects.

JSON, YAML, BSON and Messagepack object notation classes directly descends from the Object Notation classes.

For JSON:

  • TkbmMWJSONObject
  • TkbmMWJSONArray
  • TkbmMWJSONNative
For YAML:

  • TkbmMWYAMLObject
  • TkbmMWYAMLArray
  • TkbmMWYAMLNative
For BSON:

  • TkbmMWBSONObject
  • TkbmMWBSONArray
  • TkbmMWBSONNative
For Messagepack:

  • TkbmMWMessagepackObject
  • TkbmMWMessagepackArray
  • TkbmMWMessagepackNative

Since XML is a more rich notation format, it was decided to leave its SAX based parser (which the XML DOM class use) and instead add support for converting the XML DOM tree to and from the Object Notation format. This way its easy to convert between different notation formats, without sacrificing the more advanced features of our native XML parser.

The above specialized object notation classes, are in reality very very slim.
Thus the underlying TkbmMWON…. classes are sufficient for conversion. So even if one build a JSON object notation tree in code, it can still be understood by the Messagepack formatter code etc.

Generally it’s recommended using the non specialized classes instead of the specialized ones for simplicity reasons.

So lets try to build a simple object notation tree:

var
 father,child:TkbmMWONObject;
 children:TkbmMWONArray;
begin
 father:=TkbmMWONObject.Create;
 father.AsString['name']:='Joe Simpson';
 father.AsInt32['age']:=42;
 children:=TkbmMWONArray.Create;
 child:=TkbmMWONObject.Create;
 child.AsString['name']:='Joe Simpson Jr';
 child.AsInt32['age']:=12;
 children.Add(child);
 father.AsArray['children']:=children;
 ...

This basically makes a father object with name and age, and a list of children (containing only one child with name and age). The object notation tree should now look like this:

2017-02-01-01_09_51-xbox

So lets convert this to JSON:

var
 json:TkbmMWJSONStreamer;
 s:string;
begin
 json:=TkbmMWJSONStreamer.Create;
 try
  // If set to true, an exception will be raised in case a type is not supported
  // by the specific streamformat. An event also exists which one can hook
  // into, to get information about what type fails etc.
  json.ExceptOnUnknownType:=true;
  s:=json.SaveToUTF16String(father);

  // s now contains the serialized object in JSON format.
 finally
  json.Free;
 end;
end;

Resulting in s containing:

{"name":"Joe Simpson","age":42,"children":[{"name":"Joe Simpson Jr","age":12}]}

or to YAML:

var
 yaml:TkbmMWYAMLStreamer;
 s:string;
begin
 yaml:=TkbmMWYAMLStreamer.Create;
 try
  // If set to true, an exception will be raised in case a type is not supported
  // by the specific streamformat. An event also exists which one can hook
  // into, to get information about what type fails etc.
  yaml.ExceptOnUnknownType:=true;
  s:=yaml.SaveToUTF16String(father); 

  // s now contains the serialized object in YAML format.
 finally
  yaml.Free;
 end;
end;

Resulting in s containing (with linefeeds):

name: Joe Simpson
age: 42
children:
  -
    name: Joe Simpson Jr
    age: 12

BSON and Messagepack follows the same pattern as JSON and YAML.

And for XML:

var
 xml:TkbmMWDOMXML;
 s:string;
begin
 xml:=TkbmMWDOMXML.Create;
 try
  xml.LoadFromObjectNotation(father);
  xml.Top.Name:='father';
  xml.AutoIndent:=true;
  xml.AutoLineFeed:=true;
  xml.Typed:=false;
  s:=xml.SaveToString;

  // s now contains the serialized object in XML format.
 finally
  xml.Free;
 end;
end;

Resulting in s containing (with linefeeds):

<father>
 <name>Joe Simpson</name>
 <age>42</age>
 <children i="0">
  <name>Joe Simpson Jr</name>
  <age>12</age>
 </children>
</father>

In BSON, the output would be a binary value (a stream or a byte array). A Base64 encoded version of the above object notation tree would be:

XQAAAAJuYW1lAAwAAABKb2UgU2ltcHNvbgAQYWdlACoAAAAEY2hpbGRyZW4ALwAAAAMwACcAAAACbmFtZQAPAAAASm9lIFNpbXBzb24gSnIAEGFnZQAMAAAAAAAA

and a Base64 encoded Messagepack stream would look like:

g6RuYW1lq0pvZSBTaW1wc29uo2FnZSqoY2hpbGRyZW6RgqRuYW1lrkpvZSBTaW1wc29uIEpyo2FnZQw=

The Object Notation classes also supports letting the developer indicate if a property key/value pair of type TkbmMWONNative should be streamed as an attribute or an element in XML.

Further the TkbmMWONNative class supports custom values, like the BSON Decimal128 value. As no other streamers understand that value type, using custom types do limit lossless conversion between different object notation formats.

Delphi native objects can be marshalled to and from the Object Notation format, and thus also to and from the above supported specialized object notation formats formats.

Teaser : Easy REST with kbmMW and Delphi

kbmMW has for many years supported REST, but it did require some manual work in marshalling data in and out. Further the URL’s allowed for calling kbmMW’s REST interface had to be formatted fairly strictly.

Using the new kbmMW smart service which is being developed, you can extremely easily publish functions to be consumed by the public via REST via any URL you like.

A kbmMW smart service which have some functions available for calling from smart clients (a new kbmMW client type) and REST clients, could look like this:

type

 // This service is known to smart clients as SMARTDEMO
 // Methods registered for REST access will, if they
 // provide a relative path, be sub pathed under /myserver
 [kbmMW_Service('SMARTDEMO')]
 [kbmMW_Rest('path:/myserver')]
 TkbmMWCustomService2 = class(TkbmMWCustomSmartService)
 public

 // This method can be called both from REST and
 // from smart clients.
 // Its called from REST clients like this:
 // http://../helloworld
 [kbmMW_Method('HelloWorld')]
 [kbmMW_Rest('method:get, path: "/helloworld"')]
 function HelloWorld:string;

 // This method can be called both from REST and
 // from smart clients.
 // Its called from REST clients like this:
 // http://../myserver/myechostring/somestring
 // or
 // http://../myserver/echostring/somestring
 // somestring (can be anything) will be echoed back.
 // The argument AString will automatically receive
 // the value provided as somestring.
 [kbmMW_Method('EchoString')]
 [kbmMW_Rest(
 'method:get, '+
 'path: [ "echostring/{AString}","myechostring/{AString}" ]')]
 function EchoString(
  [kbmMW_Rest('value: "{AString}"')] const AString:string):string;

// This method is only available from REST calls, not
 // from smart clients. (kbmMW_Method attribute is missing)
 // It can be called from a browser like this:
 // http://.../someabspath/addnumbers?arg1=10&arg2=20
 // The functions arguments are automatically populated
 // with the arguments.
 [kbmMW_Rest('method:get, path: "/someabspath/addnumbers"')]
 function AddNumbers(
  [kbmMW_Rest('value: "$arg1", required: true')] const AValue1:integer;
  [kbmMW_Rest('value: "$arg2", required: true')] const AValue2:integer;
  [kbmMW_Arg(mwatRemoteLocation)] const ARemoteLocation:string):integer;

// This method gets and owns a TPerson instamce, which it must
 // itself free. The method is not available for REST calls.
 [kbmMW_Method]
 function StorePerson(
  [kbmMW_Arg(mwatOwnedValue)] const APerson:TPerson):integer;

 // This method is only available from smart clients.
 // It receives a TPerson instamce which is owned by the
 // caller. When called from a smart client, the kbmMW framework
 // is the owner and will automatically free it.
 [kbmMW_Method]
 function StorePerson2(const APerson:TPerson):integer;

 // This method is only available from smart clients.
 // It receives a IPerson2 interface.
 [kbmMW_Method]
 function StorePerson3(const APerson:IPerson2):integer;

 // This method can be called both from REST and
 // from smart clients.
 // The method returns a TPerson instamce which is
 // automatically freed by the kbmMW framework after marshalling.
 // (Its default to free a returned object)
 [kbmMW_Rest('method:get, path: "getperson/{id}", freeResult:true')]
 [kbmMW_Method(true)]
 function GetPerson([kbmMW_Rest('value:{id}')]id:string):TPerson;
end;

// Functions published by the service.
//------------------------------------

function TkbmMWCustomService2.HelloWorld:string;
begin
 Result:='Hello world';
end;

function TkbmMWCustomService2.EchoString(const AString:string):string;
begin
 Result:=AString;
end;

function TkbmMWCustomService2.ReverseString(
 const AString:string;
 const AClientIdentity:TkbmMWClientIdentity):string;
begin
 Result:=StrUtils.ReverseString(AString);
end;

function TkbmMWCustomService2.AddNumbers(
 const AValue1,AValue2:integer;
 const ARemoteLocation:string):integer;
begin
 Result:=AValue1+AValue2;

// In ARemoveLocation the reported remote location
 // for the client is found.
end;

function TkbmMWCustomService2.StorePerson(
 const APerson:TPerson):integer;
begin
 // The TPerson instance is owned by this function,
 // and must be manually freed.
 APerson.Free;
 Result:=0;
end;

function TkbmMWCustomService2.StorePerson2(
 const APerson:TPerson):integer;
begin
 // The TPerson instance is automatically freed
 // upon exit of this function.
 Result:=0;
end;

function TkbmMWCustomService2.StorePerson3(
 const APerson:IPerson2):integer;
begin
 // The lifetime of IPerson2 depends on reference counting.
 Result:=0;
end;

function TkbmMWCustomService2.GetPerson(id:string):TPerson;
begin
 Result:=TPerson.Create;
 Result.Name:='Sofie Mogensen';
 Result.Age:=87;
end;

initialization
 TkbmMWRTTI.EnableRTTI(TkbmMWCustomService2);

Examples on REST calls:

GET http://localhost/myserver/getperson/1

{“person”:{“Name”:”Sofie Mogensen”,”Address”:””,”Age”:87}}

GET http://localhost/someabspath/addnumbers?arg1=10&arg2=20

30

Its dead easy to create the server. Just have a central form/datamodule with a kbmMWServer instance, a transport with the new REST streamformat specified, and configured to listen on port 80, and add all the smart service units you want (like the above). Then only the following 2 lines are needed:

kbmMWServer1.AutoRegisterServices;
kbmMWServer1.Active:=true;

and you are running your REST and smart client server!

As usual all this can be combined with traditional services and clients. While REST calls results in objects automatically being streamed and unstreamed as JSON, smart clients will automatically stream and unstream object and interface instances in messagepack format, which is now also supported.