Comparing maps and arrays

In javascript it's easy to compare simple types but the same doesn't apply to maps or arrays. Let's take some examples:

var a = {
  "name": "Line",
  "points": {
    "start": {
      "x": 5,
      "y": 10
    },
    "end": {
      "x": 15,
      "y": 25
    }
  }
};

var b = {
  "name": "Line",
  "points": {
    "start": {
      "x": 5,
      "y": 10
    },
    "end": {
      "x": 15,
      "y": 25
    }
  }
}

The map a and b are almost equal but if you compare them:

> a == b
false
> 1 == 1
true
> "a" = "a"
true

The reason is that a and b are different objects despite having the same keys, strutucture and values. The only way to compare is to compare key by key and value by value. In OpenAF the function compare does just that:

> compare(a, b)
true
>
> p.points.start.x = 0
> p.points.start.y = 0
>
> compare(a, b)
false

Using the openaf console you can actually see the difference:

> diff a with b
 -      "x": 5,
 -      "y": 10
 +      "x": 0,
 +      "y": 0

Comparing arrays

What about arrays? Well, they are also objects. Alone (e.g. [1, 2, 3]) or mixed (e.g. { a: 1, b: [ 1, 2, 3]}). So you can use the compare function in the same way:

> compare([1, 2, 3], [1, 2, 3])
true
> compare([0, 1, 2, 3, 4], [1, 2, 3])
false

Applying selectors to an array

Whenever using an array of maps, and specially when analyzing them on OpenAF-console, you probably want to focus just on some map entries and not have all the other map keys on the screen. To make it easier there is a function mapArray in OpenAF for this case and others.

Let's say you have an array of maps and submaps like this:

> $rest().get("https://jsonplaceholder.typicode.com/users");
[
  {
    "id": 1,
    "name": "Leanne Graham",
    "username": "Bret",
    "email": "Sincere@april.biz",
    "address": {
      "street": "Kulas Light",
      "suite": "Apt. 556",
      "city": "Gwenborough",
      "zipcode": "92998-3874",
      "geo": {
        "lat": "-37.3159",
        "lng": "81.1496"
      }
    },
    "phone": "1-770-736-8031 x56442",
    "website": "hildegard.org",
    "company": {
      "name": "Romaguera-Crona",
      "catchPhrase": "Multi-layered client-server neural-net",
      "bs": "harness real-time e-markets"
    }
  },
  {
    "id": 2,
    "name": "Ervin Howell",
    "username": "Antonette",
    "email": "Shanna@melissa.tv",
    "address": {
[...]

Let's say that you want to compare just the name, the city on the address and the company name. With mapArray that's easy:

> mapArray( $rest().get("https://jsonplaceholder.typicode.com/users"), [ "name", "address.city", "company.name" ] )
[
  {
    "name": "Leanne Graham",
    "address.city": "Gwenborough",
    "company.name": "Romaguera-Crona"
  },
  {
    "name": "Ervin Howell",
    "address.city": "Wisokyburgh",
    "company.name": "Deckow-Crist"
  },
  {
    "name": "Clementine Bauch",
    "address.city": "McKenziehaven",
[...]

Since now you just have the fields you want you can also use table on openaf-console:

> table mapArray( $rest().get("https://jsonplaceholder.typicode.com/users"), [ "name", "address.city", "company.name" ] )
          name          | address.city |   company.name
------------------------+--------------+------------------
Leanne Graham           |Gwenborough   |Romaguera-Crona
Ervin Howell            |Wisokyburgh   |Deckow-Crist
Clementine Bauch        |McKenziehaven |Romaguera-Jacobson
Patricia Lebsack        |South Elvis   |Robel-Corkery
[...]

If you have an array on a map element you can just reference it directly:

> mapArray( someMap, ["id", "items[0].subId"])
[
  {
    "id": 1,
    "items[0].subId": 1
  },
  {
    "id": 2,
    "items[0].subId": 4
  },
  {
    "id": 3,
    "items[0].subId": 43
  }
]

Note: mapArray uses ow.obj.getPath and ow.obj.setPath internally that you can use in other situations directly.

Easily add a SSL certificate to access an URL

When accessing an URL that hosts a self-signed SSL certificate you might need to add the certificate to your's Java runtime environment trusted certificates. If you search around the internet you will find several articles explaining how to do this using Java's keytool utility (adding the certificate to the Java's keystore).

You might find a very old piece of code, posted on Sun's blog, which would connect to a host on a specific port, examine the certificate and added it to the Java's keystore. That code was packed in an OpenAF's oPack for ease of use.

Install it

To install it just execute:

$ opack install InstallCert

To use it

To use it change the current directory to the JRE/JDK's security path. This is usually on $JAVA_HOME/jre/lib/security. To use it:

$ cd $JAVA_HOME/jre/lib/security
$ opack exec InstallCert some.host:443

Note: you can use a different port (instead of 443) if needed.

During the execution you will be prompted to add the certificate to the trusted keystore by answering the number of the certificate. If needed execute more than one time to add all certificates or ensure that the certificate is now trusted (e.g. won't ask if you want to add it).

If the keystore has a different password

The code will generate a file called jssecacerts. If you already have a similar file with a different password from the default you may enter it like this:

$ cd $JAVA_HOME/jre/lib/security
$ opack exec InstallCert some.host:443 myNewPassword

Searching map keys and values

The main data structure in OpenAF is, of course, javascript maps. We already cover how to easily search on arrays using $from. But what about maps?

Let's take an example:

> var weather = $rest().get("http://www.metaweather.com/api", { location: 742676 });

The "weather" variable now should have a map with the weather forecast for Lisbon. If you inspect it has sub-arrays with the forecast for the next days, map entries to provide info about sun rise and sun set, an array with the weather sources, etc...

Searching keys

If you now the exact path it's easy to build the map "path" to get the values that you want but the OpenAF function searchKeys will actually help you find those paths. Let's try to find those sun rise and sun set entries:

> searchKeys(weather, "sun")
{
  ".sun_rise": "2019-08-27T07:00:52.671773+01:00",
  ".sun_set": "2019-08-27T20:14:52.385796+01:00"
}

The string search parameter (e.g. sun) is actually a case insensitive regular expression. And it just found you all map entries with the word "sun" anywhere. Now you know that weather.sun_rise and weather.sun_set will get you the info you need. Let's try to find now temperatures:

> searchKeys(weather, "temp")
{
  ".consolidated_weather[0].min_temp": 17.98,
  ".consolidated_weather[0].max_temp": 23.674999999999997,
  ".consolidated_weather[0].the_temp": 24.23,
  ".consolidated_weather[1].min_temp": 17.405,
  ".consolidated_weather[1].max_temp": 23.05,
  ".consolidated_weather[1].the_temp": 23.39,
  ".consolidated_weather[2].min_temp": 16.985,
  ".consolidated_weather[2].max_temp": 24.47,
  ".consolidated_weather[2].the_temp": 24.625,
[...]

Now the temperatures are found within the consolidate_weather array. So you now have a list of all the paths to get the temperatures even within an array.

Searching values

Searching keys of a map wouldn't be complete without being able to search for values. On the initial request we used the Lisbon location code "742676". Let's search for that:

> searchValues(weather, "742676")
{
  ".woeid": 742676
}

As expected the result is the path on where that value was found. Let's now use a regular expression to try to find if there are any lat/log coordinates:

> searchValues(weather, "-?\\d+\\.\\d+,-?\\d+\\.\\d+");
{
  ".parent.latt_long": "39.557919,-7.844810",
  ".latt_long": "38.725670,-9.150370"
}

Other advanced uses

Where these two functions searchKeys and searchValues really shine when you are using the openaf-console to investigate a big and/or complex javascript map. They will quickly filter and find you the data that you need to help you write your scripts.

The previous examples show you the basic use of these functions. There are other options (check, in the openaf-console, "help searchKeys" and "help searchValues") to make the search case sensitive and even to execute a callback function whenever keys or values are found (for example: search & replace).

Using an in-memory database

OpenAF comes with the H2 database embeeded. The main objective is, of course, to interact with H2 databases. But it also provides other perks (e.g. like the MVStore). One of them is an "in-memory" H2 database.

Why would you want or need an "in-memory" database? There are a lot of uses that we won't elaborate now but you can read them in wikipedia. But since H2 is also a relational database engine it can be helpful even if it just for testing.

Creating

Creating the H2 in-memory database it's easy. Just provide a name and execute:

> var db = createDBInMem("testDB");

Why a name? Because you can have several databases providing you have the memory capacity for them. The OpenAF's createDBInMem function returns an already instatiated db object that you can now use.

Using it

> db.u("CREATE TABLE test (id NUMBER(10), desc VARCHAR2(255))");
0
> db.q("SELECT * FROM test");
{
  "results": []
}

Let's insert some data:

> db.us("INSERT INTO test (id, desc) VALUES (?, ?)", [ 0, "Result 0" ]);
1
> db.q("SELECT * FROM test")
{
  "results": [
    {
      "ID": 0,
      "DESC": "Result 0"
    }
  ]
}

Let's make it a hundred dummy data records:

> var ar = []; for(var ii = 1; ii < 99; ii++) { ar.push([ ii, String("Result " + ii)]); }
98
> db.usArray("INSERT INTO test (id, desc) VALUES (?, ?)", ar)
98
> db.commit();
> 
> db.q("SELECT COUNT(1) c FROM test")
{
  "results": [
    {
      "C": "99"
    }
  ]
}

Persisting

If you exit OpenAF the in-memory database will, of course, lose all it's data. But what if you want to persist it? There are some helper functions for that: persistDBInMem and loadDBInMem:

> persistDBInMem(db, "test.sql");

Then on another OpenAF execution:

> var db = createDBInMem("testDB");
> loadDBInMem(db, "test.sql")
5
> db.q("select count(1) C from test")
{
  "results": [
    {
      "C": "99"
    }
  ]
}

Note: the generated files are SQL files. There isn't any intention of supporting large volumes of data databases.

Using OpenAF’s merge

OpenAF includes a base function to help on the task of merging maps. Let’s consider a simple example:

var source = {
   x: 1,
   y: -1
}

And you want to merge it with a map with the extra keys you need:

var extra = {
   z: -5
}

Now to merge the two maps simply execute:

> merge(source, extra)
{ 
   x: 1,
   y: -1,
   z: -5
}

Merging into arrays

Now imagine that instead of one map you have an array of maps? No problem, merge will merge with every single map part of the array of maps.

var source = [{
   x: 1,
   y: -1
}, {
   x: -5,
   y:-5
}];

var extra = { 
   z: -10
};

sprint(merge(source, extra))

// The result will be:
//[{
//   x: 1,
//   y -1,
//   z: -10
//},{
//   x: -5,
//   y: -5,
//   z: -10
//}]

Note: the keys of the second parameter map will always override the equivalent keys on the source (either in map or array mode).

Handling map keys spacing in templates

If you ever came across a similar JSON map to this:

{
   "Update"      : "2019-08-24T12:34:56.789Z",
   "Stats"       : {
      "Memory stats": {
         "Free memory" : 1234,
         "Total memory": 5678
      }
   }
}

You know that to access JSON map keys with space in javascript you need to do something like this:

print("Update: " + myMap.Update);
print("Free mem: " + myMap.Stats["Memory stats"]["Free memory"] + "MB");

To replicate in a HandleBars template you would write for the first line:

Update: {{Update}}

But, how to access the keys with space in HandleBars?

Update: {{Update}}
Free mem: {{Stats.[Memory stats].[Free memory]}}MB

So the final code would be, for example:

tprint("Update: {{Update}}", myMap);
tprint("Free mem: {{Stats.[Memory stats].[Free memory]}}MB", myMap);

Encode/Decode base64 in OpenAF

Base64 is a enconding scheme used to represent binary data in ASCII strings (using only 6-bit instead of the full 8-bit). It's used in several cases: LDAP's LDIF files character representation; encoding binary email attachments; embeeding images and fonts in HTML/CSS; avoiding delimiters been interpreted as delimiters inside a sequence of characters on a field: etc...

So, how to quickly encode or decode a string or an array of bytes to and from Base 64 in OpenAF:

> af.fromBytes2String(af.toBase64Bytes("This is a test"))
"VGhpcyBpcyBhIHRlc3Q="

> af.fromBytes2String(af.fromBase64("VGhpcyBpcyBhIHRlc3Q="))
"This is a test"

Keep in mind that the af.toBase64Bytes & af.fromBase64 return always a byte array. So in order to display it you need to use af.fromBytes2String to convert the byte array back to a string.

Of course, to convert a binary content you can just provide the array of bytes:

> af.fromBytes2String(af.toBase64Bytes(io.readFileBytes("openaf.ico")))
AAABAAQAgIAAAAEAIAAoCAEARgAAAEBAAAABACAAKEIAAG4IAQAgIAAAAQAgAKgQAACWS...

Making REST service calls from OpenAF

OpenAF includes several ways to make HTTP connections and retrieve data from them. An usually the most usefull connections are to services that expose some kind of REST API "talking" in JSON. To make it easy in OpenAF to make these calls and parse the corresponding output the shortcut $rest was created.

Simple examples

GET

Performing a simple REST GET operation:

> $rest().get("https://ifconfig.co/json");
{
  "ip": "1.2.3.4",
  "country": "Ireland",
  "country_eu": true,
  "country_iso": "IE",
  "city": "Dublin",
  "hostname": "ec2-1-2-3-4.eu-west-1.compute.amazonaws.com",
  "latitude": 53.3338,
  "longitude": -6.2488,
  "asn": "AS16509",
  "asn_org": "Amazon.com, Inc."
}

The return is a parsed json object ready to use.

POST

Performing a REST POST operation you can pass the JSON body directly (in this example the httpbin.org service will echo the post request made):

> $rest().post("https://httpbin.org/post", { a: 1, b: "xyz", c: true }).json
{
  "a": 1,
  "b": "xyz",
  "c": true
}

PUT

The same goes for the PUT verb.

> $rest().put("https://httpbin.org/put", { a: 1, b: "xyz", c: true }).json
{
  "a": 1,
  "b": "xyz",
  "c": true
}

PATCH

And the PATCH verb.

> $rest().patch("https://httpbin.org/patch", { a: 1, b: "xyz", c: true }).json
{
  "a": 1,
  "b": "xyz",
  "c": true
}

DELETE

Finally, the DELETE verb.

> $rest().delete("https://httpbin.org/delete").json
null

Passing resource/query elements

There are two ways:

1. Using the URI path (resource)

// https://some.server/restAPI/category/abc/article/1234
$rest().get("https://some.server/restAPI", { category: "abc", article: 1234 });

$rest().post("https://some.server/restAPI", { 
    article: 1234, title: "My article" 
}, { category: "abc", article: 1234 });

$rest().put("https://some.server/restAPI", { 
    article: 1234, title: "My article updated" 
}, { category: "abc", article: 1234 });

$rest().delete("https://some.server/restAPI", { category: "abc", article: 1234 });

2. Using the query string

// https://some.server/restAPI?category=abc&article=1234
$rest({ uriQuery: true }).get("https://some.server/restAPI", { category: "abc", article: 1234 });

$rest({ uriQuery: true }).post("https://some.server/restAPI", { 
    article: 1234, title: "My article" 
}, { category: "abc", article: 1234 });

$rest({ uriQuery: true }).put("https://some.server/restAPI", { 
    article: 1234, title: "My article updated" 
}, { category: "abc", article: 1234 });

$rest({ uriQuery: true }).delete("https://some.server/restAPI", { category: "abc", article: 1234 });

What if the body is URL encoded?

When you need the request body to be "application/x-www-form-urlencoded" and instead of "application/json" you can use the option urlEncode=true.

> $rest({ urlEncode: true }).post("https://httpbin.org/post", { a: 1, b: "xyz" }).form
{
  "a": "1",
  "b": "xyz"
}

This was a simple introduction to the $rest shortcut. Check all the available options (e.g. authentication, timeout, exception control, etc...) executing "help $rest.get" from an openaf-console.

Protecting from system exit

Whenever running a script you may want to abort/stop the current execution completely exiting the OpenAF process. You can do that using the function exit:

exit(0);
// Exits the current execution immediatelly with exit code 0

exit(-1);
// Exits the current execution immediatelly with exit code -1

But what if you are calling that script from the openaf-console (using the load function) or similar and you don't want the entire Java process to "die"? You can use the af.protectSystemExit function for that:

> af.protectSystemExit(true, "No can do!: ");
> exit(0);
-- No can do!: 0
> exit(-10);
-- No can do!: -10

Everytime the script tries to end processing by means of a system exit a javascript execption will be raised with the provided text appended with the corresponding exit code.

If you want to "disarm" this protection:

> af.protectSystemExit(false);
> exit(0);
$

Note: this protection uses the Java security manager. So even java system exit code will be affected by this "protection".

Setting DB auto commit

By default OpenAF opens all database connections in "autocommit=false" mode. But in some special cases you might need to turn it on. Let's see one of those cases:

Creating a PostgreSQL tablespace example

If you open a PostgreSQL connection and try to create a tablespace "TEMP" as you would usually do, this will happen:

> var db = new DB("jdbc:postgresql://some.host:5432/postgres", "adminUser", "adminPass");
> db.u("create tablespace temp location '/data'");
-- JavaException: org.postgresql.util.PSQLException: ERROR: CREATE TABLESPACE cannot run inside a transaction block

That's because PostgreSQL considers anything with "autocommit=false" to be a transaction block.

How to set auto commit to on?

To bypass this situation, on a dedicated DB connection, execute the following:

> var db = new DB("jdbc:postgresql://some.host:5432/postgres", "adminUser", "adminPass");
> db.getConnect().setAutoCommit(true);
> db.u("create tablespace temp location '/data'");
1

The getConnect method will return you the JDBC connection instance object which, among others, allows you to execute the setAutoCommit method. That's enough to then execute the "create tablespace" statement without being in a transaction block.

Check if an email address is valid for sending emails

Whenever you have a list of email addresses you might wonder if the email address provided really exists or if it's a "dummy" email. Since internet email does not provide any delivery guarantees nor there exists any reply standard (or obligation) from target domains to reply back that the email address does not exist what can you check to clean cases like "my.name@you.wont.find.me" automatically?

The options are:

• Find if the domain really exists
• Find if the domain has any email servers associated

Both can easily be checked by performing DNS lookups. Most operating systems provide the command "nslookup" to easily lookup this information.

Note: There are several types of entries in DNS records. For example the entry type "A" means the IP4 address associated with the domain record while "MX" means the email server associated with the domain record.

An easy way to implement this is to use the Google Public DNS-over-HTTPS service. This allows you to easily retrieve the info needed throw a public REST call.
Here is an example of usage in OpenAF scripting:

function(checkEmail) {
   var dom = email.split(/@/)[1]; 
   var res = $rest({ uriQuery: true })
             .get("https://dns.google.com/resolve", { 
                name: "gmail.com", 
                type: 255 
             });

   return { 
      exists: isDef(res) && isDef(res.Answer),
      hasMXRecords: (isDef(res.Answer)) 
                      ? $from(res.Answer).equals("type", 15).any()
                      : false
   } 
}

Executing this function for some examples (valid at the time of writing):

// goo.gl is a google url shortener service, it exists but no emails can't be sent to it
> checkEmail("abc@goo.gl") 
{
   "exists": true,
   "hasMXRecords": false
}

// gmail.com exists and emails can be sent to it
> checkEmail("abc@gmail.com") 
{
   "exists": true,
   "hasMXRecords": true
}

// miguel.com is a domain bought to be resold but doesn't have a site nor email can be sent to it
> checkEmail("abc@miguel.com")
{
   "exists": false,
   "hasMXRecords": false
}

// nuno.com is a japanese textil company, so the domain exists and emails can be sent to it
> checkEmail("nuno@nuno.com")
{
   "exists": true,
   "hasMXRecords": true
}

// finally, my.name@you.wont.find.me (yes, it exists and it's a site)
> checkEmail("my.name@you.wont.find.me")
{
   "exists": true,
   "hasMXRecords": false
}

Sending emails from oJob

Sending emails from oJob is made easy due to the oJobEmail.yaml include from the oJob-common opack. As we other common ojobs the idea is to be easy to use. Let's look at a very simple example:

ojob:
  opacks:
    - oJob-common

include:
  - oJobEmail.yaml

jobs:
  # -----------------------
  - name: Send a test email
    to  : oJob Send email
    args:
      from       : "my.email@gmail.com"
      server     : "smtp.gmail.com"         # for example
      credentials:
        user: "my.email@gmail.com"
        pass: "myAppPassword"
      useSSL     : true
      useTLS     : true
      to         : "someone@somewhere.com"
      subject    : "My test email"
      output     : "This is a test email to test sending emails from oJob"
      debug      : false

todo:
  - Send a test email

This example will send an email from my.email@gmail.com to someone@somewhere.com with a test subject a email message.

SMTP server settings

Depending on the SMTP email servers you might need to use different settings:

Setting	Type	Default	Description
useSSL	boolean	false	If the SMTP connection should use SSL (can use also TLS).
useTLS	boolean	false	If the SMTP connection should use TLS (can use also SSL).
port	number	25/587/465	Specify the port to contact the SMTP server.
server	string	n/a	The server address.
credentials	map	n/a	A map with user and pass to use to authenticate on SMTP servers.
debug	boolean	false	Very useful to debug SMTP connections by showing all the communication between oJob and the SMTP server.

Email basic settings

For the email specifics you can see in the example the use of the from, to, subject and output fields. The other possible fields are also the obvious:

Setting	Type	Description
from	string	A string representing the "from" address.
to	string/array	An array or a string of "to" addresses.
cc	string/array	An array or a string of "cc" addresses.
bcc	string/array	An array or a string of "bcc" addresses.
subject	string	The text subject of the email message.
output	string	The text message of the email message.
isHTML	boolean	Specifies, if true, that the output is in HTML format.
altOutput	string	If isHTML = true and the end client doesn't support HTML this will be the alternative output to show.

Sending attachments

To send any file as an attachment just use "addAttachments":

      addAttachments:
        - name    : file.txt
          file    : /some/path/file.txt
        - name    : secondFile.txt
          file    : /some/path/secondFile.txt

Adding images to HTML emails

You have three options:

Embed image URLs

      isHTML   : true
      output   : "<html>...<img src=\"cid:myimage\"/>...</html>"
      embedURLs:
        - name: myimage
          url : https://some.server/some/image.jpg

Embed image files

      isHTML        : true
      output        : "<html>...<img src=\"cid:myimage.png\"/>...</html>"
      embedFiles:
        - name    : myimage.png
          file    : /some/path/myimage.png

Automatic embedding images and reference them by URL

      isHTML   : true
      output   : "<html>...<img src="https://some.server/some/image.jpg"/>...</html>"
      addImages:
        - https://some.server/some/image.jpg

OpenAF-console color formatting

When using the OpenAF-console if the terminal is ANSI capable some of the output will be "colored" to improve readability:

But the same on dark mode might be harder to read:

The current colors used in the OpenAF are defined in the internal variable "__colorFormat":

> __colorFormat
{
  "key": "BOLD,BLACK",
  "number": "GREEN",
  "string": "CYAN",
  "boolean": "RED",
  "default": "YELLOW"
}

You can check all the available attributes & colors executing "help ansiColor". You can change any color directly on the __colorFormat map. If you want to make your changes permanent you can add them to the ~/.openaf-console_profile file.
So let's change and see the diference:

> __colorFormat.key = "BOLD,WHITE"

Much easier to read. Of course, it all depends on your personal preferences:

{
  "key": "UNDERLINE,WHITE",
  "number": "BG_WHITE,GREEN",
  "string": "BG_WHITE,BLUE",
  "boolean": "BG_WHITE,RED",
  "default": "BG_WHITE,YELLOW"
}

oJob one-liners

On the article Micro remote HTTP file browser we described a fast way to build a micro HTTP server serving files from the current folder. But some (myself included) need faster ways to do it instead of creating an oJob file and executing it.

So doing the same thing on one line that you can just copy+paste whenever you need can be very pratical. But you will be sacrifying readability by anyone else and you. Since oJobs can be used in YAML or JSON this enables you to build a YAML oJob file and then convert it to JSON string where initilization parameters can be easily change. Additionally you can convert back to YAML very easily.

Example

Let's use the micro remote HTTP file browser example. Since YAML provides some features that JSON doesn't (like anchors) the original YAML needs some adaptation:

httpd.yaml

init:
  port: 8080
  path: "."
  uri : "/"

ojob:
  daemon    : true
  sequential: true
  opacks    :
    - oJob-common

include:
  - oJobHTTPd.yaml

todo:
  - name: Init
  - name: HTTP Start Server
    args: "({ port: global.init.port, mapLibs: true })"
  - name: HTTP File Browse
    args: "({ port: global.init.port, path: global.init.path, uri: global.init.uri })"

jobs:
  # ----------
  - name: Init
    exec: "global.init = args.init;"

Convert to a JSON one-liner

Then to convert this YAML file into a JSON string use openaf-console and execute:

> print(stringify(io.readFileYAML("httpd.yaml"), void 0, ""))

{"init":{"port":8080,"path":".","uri":"/"},"ojob":{"daemon":true,"sequential":true,"opacks":["oJob-common"]},"include":["oJobHTTPd.yaml"],"todo":[{"name":"Init"},{"name":"HTTP Start Server","args":"({ port: global.init.port, mapLibs: true })"},{"name":"HTTP File Browse","args":"({ port: global.init.port, path: global.init.path, uri: global.init.uri })"}],"jobs":[{"name":"Init","exec":"global.init = args.init;"}]}

Use the JSON one-liner

Then to use this JSON as an one-liner:

> oJobRun( /* one-liner begin */  {"init":{"port":8080,"path":".","uri":"/"},"ojob":{"daemon":true,"sequential":true,"opacks":["oJob-common"]},"include":["oJobHTTPd.yaml"],"todo":[{"name":"Init"},{"name":"HTTP Start Server","args":"({ port: global.init.port, mapLibs: true })"},{"name":"HTTP File Browse","args":"({ port: global.init.port, path: global.init.path, uri: global.init.uri })"}],"jobs":[{"name":"Init","exec":"global.init = args.init;"}]}  /* one-liner end */ );

To stop just hit "Ctrl-C".

Making changes

If you need to change the folder, port or uri used it's easy to just direclty change on the first init map:

{"init":{"port":8080,"path":".","uri":"/"},

To convert it back to YAML:

io.writeFileYAML("httpd.yaml", /* one-liner begin */  {"init":{"port":8080,"path":".","uri":"/"},"ojob":{"daemon":true,"sequential":true,"opacks":["oJob-common"]},"include":["oJobHTTPd.yaml"],"todo":[{"name":"Init"},{"name":"HTTP Start Server","args":"({ port: global.init.port, mapLibs: true })"},{"name":"HTTP File Browse","args":"({ port: global.init.port, path: global.init.path, uri: global.init.uri })"}],"jobs":[{"name":"Init","exec":"global.init = args.init;"}]}  /* one-line end */ );

Note: since you lose comments we would recommed you to just use the original YAML file.

Timing executions on OpenAF console

When profilling your script or functions you created it's usually helpfull to quickly understand how many ms or seconds it took to execute. Using the OpenAF console you can use the "time" command to provide you the execution time of every console line executed:

> time
-- Timing of commands is enabled.
> sleep(500)
-- Elapsed time: 0.51s

Analysing results

Consider the following example:

> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.045s
> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.028s
> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.024s

"Why the first test was slower than the other two tests?" Whenever analysing execution time you need to consider several factors among them:

• OpenAF uses the Rhino javascript engine that, by default in OpenAF, will interpret the javascript code and then compile it "on-the-fly" to java bytecode. Subsequent calls will then use the compiled code.
• The are always several levels of caching. In this case the operating system might cache the results of listing details about a folder so subsequent calls will be faster.
• It's Java underneath so the Java Gargabe Collector might influence results.
• The current machine load can also influence the execution time.

Adivces

The generic advices, given the factors that might influence execution, are:

• Always time execution more than one time. Exit and reopen the openaf-console, if needed, to understand the time it takes to "heat up".
• Keep an eye on the current machine load. Make sure you only make comparitions with similar general loads.
• Make sure there is enough memory for the openaf-console Java process and the execution you are profiling.

If you want to keep an eye on the memory you can also execute the "one-line":

> ow.loadFormat();
> watch var rm = java.lang.Runtime.getRuntime(); templify("{{free}}/{{total}} ({{max}})", { free: ow.format.toBytesAbbreviation(rm.freeMemory()), total: ow.format.toBytesAbbreviation(rm.totalMemory()), max: ow.format.toBytesAbbreviation(rm.maxMemory()) })

Executing now the same calls you can now see the memory evolution ("free/total (max)"):

> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.033s
[ "48.2 MB/59.5 MB (879 MB)" ]
> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.024s
[ "45.2 MB/59.5 MB (879 MB)" ]
> var o = io.listFiles("/usr/bin")
-- Elapsed time: 0.022s
[ "42.1 MB/59.5 MB (879 MB)" ]

How to turn it off

To turn off the timming of command executions and the memory evolution watch command just execute:

> time
-- Timing of commands is disabled.
> watch off
>

You can also add time specific points in an existing script using functions like now(). For even more functionality the ow.test library provides more tools for reporting

Calling an AWS Lambda function

If you have an AWS Lambda function that you need to call from OpenAF you can use the AWS oPack. You can call it from within an AWS network or from the Internet (e.g. with the appropriate permissions). To install the AWS oPack execute:

$ opack install aws

To use the AWS functionality programmatically you will need an AWS API KEY and an AWS API Secret (check the setup instructions in AWS). Keep in mind that the API Key + Secret you will need to have the necessary permissions to execute AWS Lambda functions.

Example

Let's call a simple AWS Lambda function:

loadLib("aws.js");
var aws = new AWS(apiKey, apiSecret); // replace or define string variables with the corresponding AWS API Key & Secret

var res = aws.LAMBDA_Invoke("eu-west-1", "test");
sprint(res); // Print the result

// The variable res will have something similar to:
// {
//  "statusCode": 200,
//  "body": "\"Hello from Lambda!\""
//}

To pass arguments just add the input map:

loadLib("aws.js");
var aws = new AWS(apiKey, apiSecret); // replace or define string variables with the corresponding AWS API Key & Secret

var res = aws.LAMBDA_Invoke("eu-west-1", "addAPlusB", { a: 2, b: 3 });
sprint(res); // Print the result

// The variable res will have something similar to:
// {
//    "a": 2,
//    "b": 3,
//    "result": 5
// }

Calling an AWS Lambda asynchronously

To call an AWS Lambda asynchronously continuing the OpenAF execution without waiting for a result just use the function LAMBDA_InvokeAsync:

loadLib("aws.js");
var aws = new AWS(apiKey, apiSecret); // replace or define string variables with the corresponding AWS API Key & Secret

aws.LAMBDA_InvokeAsync("eu-west-1", "test");

It's also possible to use OpenAF as an AWS lambda language using the OpenAFLambdaLayers opack.

Generating SQL with OpenAF templates

Since OpenAF is Java and Javascript it can take advantage of the two languages. The javascript Handlebars library comes already bundled in OpenAF and provides the ability to create any kind of text templates which can be reused and changed without any code changes to a script.

Simple example

To show this let's take the hypothetical example of creating several reference tables (for the sake of space on the article we are going to consider that the fields are fixed). Handlebars will render any template using a javascript object. So let's create one for our example:

tables.json

{
  "appUser": "PROJ_APP",
  "datUser": "PROJ_DAT",
  "tablespace": "PROJ_TAB",
  "tables": [
    {
      "tableName": "tab_1"
    },
    {
      "tableName": "tab_2"
    },
    {
      "tableName": "TAB_3"
    }
  ]
}

So we are specifying the application (app) and data (dat) users to use, the tablespace to use and the table names. Now let's create a Handlebars template file (hbs):

table_creation.hbs

-- Creating tables
--

{{#each TABLES}}
-- Table for {{../datUser}}.{{tableName}}
CREATE TABLE {{../datUser}}.{{tableName}} (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE {{../tablespace}}
{{/each}}

Looking at the JSON object this template is iterating on the tables array, getting the tableName. datUser and tablespace are gather from the parent to the tables array.

And let's create the OpenAF script to bundle everything:

ow.loadTemplate();
print(ow.template.parseHBS("table_creation.hbs", io.readFile("tables.json"));

And that's it, when you run it you will get this:

-- Creating tables
--

-- Table for PROJ_DAT.tab_1
CREATE TABLE PROJ_DAT.tab_1 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.tab_2
CREATE TABLE PROJ_DAT.tab_2 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.TAB_3
CREATE TABLE PROJ_DAT.TAB_3 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB

But if we are creating tables in a data schema for which we want to create synonyms on the application schema. Well, now we just need to change the template for that:

table_creation.hbs

-- Creating synonyms
--

{{#each TABLES}}
-- Synonym for {{../appUser}}.{{tableName}}
CREATE OR REPLACE SYNONYM {{../appUser}}.{{tableName}} FOR {{../datUser}}.{{tableName}};
GRANT ALL ON {{tableName}} TO {{../appUser}} WITH GRANT OPTION;
{{/each}}

-- Creating tables
--

{{#each TABLES}}
-- Table for {{../datUser}}.{{tableName}}
CREATE TABLE {{../datUser}}.{{tableName}} (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE {{../tablespace}}
{{/each}}

And execute the unchanged OpenAF script:

-- Creating synonyms
--

-- Synonym for PROJ_APP.tab_1
CREATE OR REPLACE SYNONYM PROJ_APP.tab_1 FOR PROJ_DAT.tab_1;
GRANT ALL ON tab_1 TO PROJ_APP WITH GRANT OPTION;
-- Synonym for PROJ_APP.tab_2
CREATE OR REPLACE SYNONYM PROJ_APP.tab_2 FOR PROJ_DAT.tab_2;
GRANT ALL ON tab_2 TO PROJ_APP WITH GRANT OPTION;
-- Synonym for PROJ_APP.TAB_3
CREATE OR REPLACE SYNONYM PROJ_APP.TAB_3 FOR PROJ_DAT.TAB_3;
GRANT ALL ON TAB_3 TO PROJ_APP WITH GRANT OPTION;

-- Creating tables
--

-- Table for PROJ_DAT.tab_1
CREATE TABLE PROJ_DAT.tab_1 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.tab_2
CREATE TABLE PROJ_DAT.tab_2 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.TAB_3
CREATE TABLE PROJ_DAT.TAB_3 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB

Using helpers

We could stop the example but if you notice some table names are not all upper cases. And we like our generated SQL to be pretty :) To do that we just make some slights changes to the OpenAF script to include a helper function:

ow.loadTemplate();
ow.template.addHelper("upper", function(aString) { return aString.toUpperCase();})
print(ow.template.parseHBS("table_creation.hbs", io.readFile("tables.json"));

and use the function inside the template:

table_creation.hbs

-- Creating tables
--

{{#each TABLES}}
-- Table for {{../datUser}}.{{upper tableName}}
CREATE TABLE {{../datUser}}.{{UPPER tableName}} (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE {{../tablespace}}
{{/each}}

The result:

-- Creating tables
--

-- Table for PROJ_DAT.TAB_1
CREATE TABLE PROJ_DAT.TAB_1 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.TAB_2
CREATE TABLE PROJ_DAT.TAB_2 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.TAB_3
CREATE TABLE PROJ_DAT.TAB_3 (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE PROJ_TAB

Using conditions

Well, not all the tables will be equally created. We might not want Oracle logging for some. Let's note that on the json data:

tables.json

{
  "appUser": "PROJ_APP",
  "datUser": "PROJ_DAT",
  "tablespace": "PROJ_TAB",
  "tables": [
    {
      "tableName": "tab_1"
    },
    {
      "tableName": "tab_2",
      "nologging": true
    },
    {
      "tableName": "TAB_3"
    }
  ]
}

And change the template:

table_creation.hbs

-- Creating tables
--

{{#each TABLES}}
-- Table for {{../datUser}}.{{tableName}}
CREATE TABLE {{../datUser}}.{{tableName}} (col1 NUMBER(8), col2 VARCHAR(500)) {{#if nologging}}NOLOGGING{{/IF}} TABLESPACE {{../tablespace}}
{{/each}}

Executing the unchanged script:

-- Creating tables
--

-- Table for PROJ_DAT.tab_1
CREATE TABLE PROJ_DAT.tab_1 (col1 NUMBER(8), col2 VARCHAR(500))  TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.tab_2
CREATE TABLE PROJ_DAT.tab_2 (col1 NUMBER(8), col2 VARCHAR(500)) NOLOGGING TABLESPACE PROJ_TAB
-- Table for PROJ_DAT.TAB_3
CREATE TABLE PROJ_DAT.TAB_3 (col1 NUMBER(8), col2 VARCHAR(500))  TABLESPACE PROJ_TAB

Using template parts for increase reusability

Well this way we are going to end with lots of hbs files and little reusability besides copy+paste. Can we solve that? Yes, we case use partials. So let's create two sub templates for synonyms and tables:

table_creation.hbs

-- Creating tables for {{for}}
--

{{#each TABLES}}
-- Table for {{../datUser}}.{{upper tableName}}
{{#if NUMBER}}
CREATE TABLE {{../datUser}}.{{UPPER tableName}} (col1 NUMBER(8), col2 NUMBER(25)) TABLESPACE {{../tablespace}}
{{ELSE}}
CREATE TABLE {{../datUser}}.{{UPPER tableName}} (col1 NUMBER(8), col2 VARCHAR(500)) TABLESPACE {{../tablespace}}
{{/IF}}
{{/each}}

Now one for the synonyms:

syn_creation.hbs

-- Creating synonyms for {{for}}
--

{{#each TABLES}}
-- Synonym for {{../appUser}}.{{upper tableName}}
CREATE OR REPLACE SYNONYM {{../appUser}}.{{UPPER tableName}} FOR {{../datUser}}.{{UPPER tableName}};
GRANT ALL ON {{UPPER tableName}} TO {{../appUser}} WITH GRANT OPTION;
{{/each}}

And a main template to refer to these two sub templates (partials):

sql_creation.hbs

{{> synonyms FOR="ref tables"}}
{{> TABLES   FOR="ref tables"}}

Now we just need to alter the OpenAF script to register the sub templates (partials):

ow.loadTemplate();
ow.template.addHelper("upper", function(aString) { return aString.toUpperCase();})
ow.template.addPartial("tables", io.readFileString("table_creation.hbs"));
ow.template.addPartial("synonyms", io.readFileString("syn_creation.hbs"));
print(ow.template.parseHBS("sql_creation.hbs", io.readFile("tables.json"));

And we are done. We now can improve the synonyms and table creation separately and reuse those for different proposes while keep the reference to the same template file.

What if we want to generate something else? Is this only for SQL?

Well, it's text based. Originally Handlebars is used for web templating parsing templates using javascript. So just change the template:

SQL generation report
---------------------

Using the schemas:
- DAT = {{datUser}}
- APP = {{appUser}}

and the tablespace {{tablespace}}, the following reference Oracle objects DDL were generated:

{{#each tables}}
- The {{#if number}}number {{/if}}reference table {{upper tableName}} for the schema {{../datUser}} and tablespace {{../tablespace}}.
- The synonym from {{../datUser}}'s {{upper tableName}} for {{../appUser}}.
{{/each}}

and it's done:

SQL generation report
---------------------

Using the schemas:
- DAT = PROJ_DAT
- APP = PROJ_APP

And tablespaces:
- LARGE = PROJ_TAB

The following reference Oracle objects DDL were generated:

- The reference table TAB_1 for the schema PROJ_DAT and tablespace PROJ_TAB.
- The synonym from PROJ_DAT's TAB_1 for PROJ_APP.
- The reference table TAB_2 for the schema PROJ_DAT and tablespace PROJ_TAB.
- The synonym from PROJ_DAT's TAB_2 for PROJ_APP.
- The reference table TAB_3 for the schema PROJ_DAT and tablespace PROJ_TAB.
- The synonym from PROJ_DAT's TAB_3 for PROJ_APP.

Where can I learn more about Handlebars?

There is more functionality available including pre-compiling templates for performance (ow.template.compile, ow.template.execCompiled, ow.template.loadCompiledHBS and ow.template.saveCompiledHBS). There are several examples through the internet. But you can start with:

• Expressions (escaping, whitespace control, helpers, ...): http://handlebarsjs.com/expressions.html
• Partials (dynamic partials, block partials, inline partials): http://handlebarsjs.com/partials.html
• Built-in helpers (if, unless, each, with, lookup, log, ...): http://handlebarsjs.com/builtin_helpers.html
• Block helpers: http://handlebarsjs.com/block_helpers.html

Converting numbers

Is there a way, in OpenAF, to convert a decimal number to hex? Or to binary? Or to octal? Yes, there is on the ow.format library.
Start by loading the library:

> ow.loadFormat();

Converting from hexadecimal to decimal:

> ow.format.fromHex("1F78")
8056

Converting from decimal to hexadecimal:

> ow.format.toHex(8056)
"1f78"

Converting from decimal to octal:

> ow.format.toOctal(8056)
"17570"

Converting from octal to decimal:

> ow.format.fromOctal(17570)
8056

Converting from decimal to binary:

> ow.format.toBinary(12345)
"11000000111001"

Converting from binary to decimal:

> ow.format.fromBinary("11000000111001")
12345

And that's it. For conversions between arrays of bytes and the corresponding hexadecimal representation there are also functions like ow.format.string.toHex and ow.format.string.toHexArray.

Executing commands remotely via SSH

All the functionality to access SSH & SFTP is encapsulated within the OpenAF's SSH plugin. You can use it directly but it might be hard to keep up with methods and function parameters. On oJob definitions it's easy using the oJob-common functionality. Directly on OpenAF code to make it easy to use and read there is a shortcut: $ssh

The $ssh is a function that receives the details on how to access a host via SSH as a map. The returned object can then be used to chain command to be executed remotely and finally execute them returning an array of maps with the executions result. Let's check a simple example:

$ssh({
   host: "some.host",
   login: "aUser",
   pass: "aPass"
})
.sh("id")
.get();
/*[
  {
    "stdout": "uid=12345(user) gid=500(users) groups=500(users)",
    "stderr": "",
    "exitcode": 0,
    "cmd": "id"
  }
]*/

Why is the get function returning an array? Can a private key be used? What if the SSH server has  different port? How do I provide stdin contents? Let's take it into parts.

Connection options

In the previous example we used host, login and pass but there are more possible options. Some of them have default values like port = 22. Here is the list (you can also check it out in openaf-console executing "help $ssh.get"):

• host - The SSH host address
• port - The SSH port (defaults to 22)
• login - The login username (encrypted or not)
• id - The filepath to a SSH identity key file
• pass - The password of the login or the identity key
• timeout - The connection timeout in ms
• compress - A boolean determining if the connection should be compressed or not

It's also possible to write all these options in a single url:

ssh://[login]:[pass]@[host]:[port]/[id]?timeout=[timeout]&compression=[compression]

The execution function

The main execution function is "sh". This function receives a command as the first argument and a stdin as a second argument. They can be chained to a reasonable amount of calls that will be executed sequentially:

$ssh("ssh://aUser:aPass@somehost")
.sh("cd /some/dir && ./start.sh")
.sh("perl", "print(123)")
.get();

Note: despite being able to chain execution commands they are still "independent". Check in the example how you can change directory and then execute a command.

Retrieve results

To execute the chained commands there are two options: "get" and "exec". "get" will pack all the executions into an array of maps, each map with the corresponding command execution stdout, stderr and exitcode.  But the resulting map is only returned upon the end of the commands chain execution. Using ".exec" will still return the map on the end of the execution but any stdout and stderr output will be redirected to the scripts stdout and stderr.

If you have several $ssh entries in your script executing remote commands that return a lot of lines you might need "line labels" to understand them. This can easily be achieve using the ".cb" function (callback):

> ow.loadFormat();
> $ssh("ssh://aUser:aPass@somehost").cb(ow.format.streamSHPrefix("somehost")).sh("ls").exec()
/*
[somehost] Mail
[somehost] bin
[somehost] tmp
*/

Accessing environment variables

If you previously wrote any type of script you probably, sooner or later, want to access environment variables. OpenAF is no different and access to the operating system environment variables it's very easy with getEnv and getEnvs functions:

> getEnvs()
{
   "PATH": "/some/dir:/usr/bin",
   "SHELL": "/bin/bash",
   "CUSTOM": "something",
//...

getEnvs will retrieve a map of the current environment variables. To retrieve a specific environment variable you can directly access the map:

> getEnvs()["CUSTOM"]
"something"

or use getEnv directly:

> getEnv("CUSTOM")
"something"

Using the OpenAF's SVN plugin

There is a built-in SVN client (plugin) in OpenAF that doesn't require any other installed SVN binary to run. Below some examples of commands you can use.

Before using it you have to load the plugin:

plugin("SVN");

Checking out from SVN

You can check out without authenticating yourself (if allowed by the SVN server via HTTP):

var svn = new SVN("http://my.svn.server/svn/nAttrMon/trunk");
svn.checkout("nAttrMon") // "102"

And that's it. The folder “nAttrMon” should have been created on the current folder with a checkout from nAttrMon's current trunk. The checkout operation ends indicating the revision retrieved (in this case 102).

You can check out authenticating yourself:

var svn = new SVN("svn+ssh://my.svn.server/v1/svn/repos/project", "myuser", "mypass");
svn.checkout("."); // "1002"

Note: When providing the password to authenticate you can use also use an encrypted text.

Updating your working copy

To update your working copy (using the nAttrMon's example):

svn.update(["config/inputs", "config/outputs"], "HEAD")

This will update only the config/inputs and config/outputs folders. You may optionally also provide which revision you want (in this case the HEAD for the latest).

Commit changes and new files

Let's say you created a new input and output and you wish to add it to the input.disabled and output.disabled examples folders in nAttrMon:

var svn = new SVN("http://my.svn.server/svn/nAttrMon/trunk", "myuser", "mypass");
svn.add([
  "config/input.disabled/myAwesomeInput.js",
  "config/output.disabled/myAwesomeOutput.js"]);
svn.commit(
  ["config/input.disabled", "config/output.disabled"], 
  "My new awesome nAttrMon input and output plugs! and some bug corrections..."
);

First, before any commit you need to authenticate yourself. Then if you are adding new files you need to use svn.add and provide an array of the new files you are adding. Then finally you can commit providing a list of folders that contain new or changed files.

Additional tricks

You can retrieve just the files you need

You don't need to checkout an entire folder if you just need to files. For example:

var svn = new SVN("http://my.svn.server/svn/nAttrMon/trunk/config/inputs.disabled");
svn.update(["00.doc.js", "00.init.js"]);

Will retrieve just the 00.doc.js and 00.init.js files from the trunk/config/inputs.disabled SVN repository folder.

Revert a changed file

You aren't sure you changed a file on your working copy directory, no problem:

svn.revert(["00.init.js"]);

Note: Most SVN functions accept file path arrays meaning it could be just the file path to a file or it can be for an entire folder (for example: svn.update[“.”]).

Executing code when an OpenAF script execution is terminated

Usually there is a need to execute specific code whenever a script is ending/shutting down. Either by normal or forced termination. Examples of this are closing connected sessions to other servers (e.g. databases), deallocate used resources, etc...

In OpenAF this can be achieved using addOnOpenAFShutdown:

addOnOpenAFShutdown(function() {
   // Some closing code
   // ...
});

The following example closes the created mini web server upon normal or forced termination:

ow.loadServer();
 
// Setting up
!pidCheckIn("testserver.pid") && !log("Already running") && exit(0);
// Add code to execute on termination
addOnOpenAFShutdown(function() {
   log("Stopping...");
   hs.stop();
   log("Done!");
});
 
// Starting the httpd server
log("Starting...");
 
var hs = ow.server.httpd.start(8888);
hs.add("/", function(r) {
   return hs.replyOKText("Current date: " + new Date());
});
 
log("Ready");
 
// Converting the script into a daemon so that it doesn't terminate upon Ctrl-C or similar
ow.server.daemon();

Do note that every call to addOnOpenAFShutdown will add a new hook. Upon termination, the last added hook will be the first to be executed and backwards to the first added.

Casting values in PostgreSQL and H2 for OpenAF

When performing specific queries in PostgreSQL and H2 although some results are clearly numeric that are returned as strings. This might affect scripts that previously accessed Oracle and now, accessing using the same query in PostgreSQL and H2, have a different behavior.

Example with PostgreSQL

Let's start with a quick PostgreSQL example:

> var db = new DB("jdbc:postgresql://127.0.0.1/postgres", "postgres", "admin")
> db.q("select count(1) from (select 1 a) as a")
{
  "results": [
    {
      "count": "1"
    }
  ]
}

The "count" value is returned as a string despite that count is always numeric. To solve this you can cast the specific column on the SQL select:

> db.q("select cast(count(1) as integer) from (select 1 a) as a")
{
  "results": [
    {
      "count": 1
    }
  ]
}
>

Another option is to cast in javascript:

> var res = db.q("select count(1) from (select 1 a) as a")
> var c = Number(res.results[0].count);
1

Example with H2

The same example but with H2:

> var db = createDBInMem();
> db.q("select count(1) abc from ( select 1 abc, 'cenas' xpto from dual )")
{
  "results": [
    {
      "ABC": "1"
    }
  ]
}

The count field 'abc' is returned as string. But we can cast in the same way it has handled in PostgreSQL:

> db.q("select count(1) abc from ( select 1 abc, 'cenas' xpto from dual )")
{
  "results": [
    {
      "ABC": 1
    }
  ]
}

Another options is always to cast it in javascript:

> var db = createDBInMem();
> var res = db.q("select count(1) abc from ( select 1 abc, 'cenas' xpto from dual )")
> var c = Number(res.results[0].ABC);
1

Note: For date types ensure that you have executed db.convertDates(true) to ensure the native conversion in Java.

How to use OpenAF from Notepad++

It's possible to use Notepad++ to run and debug OpenAF scripts. To set it up just follow the steps:

1. Download and install (if don't have) the NPPExec plugin for Notepad++

   set hglt1 = ERROR [%ABSFILE%] -- *, line: %LINE%, column: %CHAR%
   c:\apps\openaf\openaf.bat -s -i script -f "$(FULL_CURRENT_PATH)"

2. Inside Notepad++ go to menu: Plugins → NPPExec → Execute…
1. Enter a similar command (correcting to your OpenAF script path):

   set hglt1 = ERROR [%ABSFILE%] -- *, line: %LINE%, column: %CHAR%
   c:\apps\openaf\openaf.bat -s -i script -f "$(FULL_CURRENT_PATH)"
   

2. You can save it as OpenAF.
3. Inside Notepad++ go to menu: Plugins → NPPExec → Console Output Filters…
1. In the HighLight tab, select the first mask and enter the following text:

   $(hglt1)

2. Enter FF on the Red component
3. Check the [B]old checkbox

Mask strings are limited to a few characters in NPP, hence the need to set the $(hglt1) variable on execution.

Now to execute you can hit Ctrl-F6 (to execute immediately) or F6 (to edit the command):

Double-clicking on the error will take to the line where the error occurred.

On the NPPExec plugin menu, click on “Save all files on execute”, so you don't have to save your script before executing it.

Adding color to OpenAF scripts

Black and white scripts can be tedious and many times when something errors out and you are tired you may not even notice it. So color not only helps to make it more "colourful" but also helps errors and warnings to stand out from the rest of the logging or printing.

In OpenAF, if the current terminal supports ANSI color you can use the ansi* functions in any script. These functions will detected if ANSI color is supported and return the proper escape sequences to produce "color".

Here is a quick description of the 3 main functions:

• ansiStart() - Detects and prepares to output ansi color escape sequences.
• ansiStop() - Stops the output of ansi color escape sequences.
• ansiColor(aAnsiSetting, aString, force) - Returns aString within the appropriate ANSI color escape sequences for the provide aAnsiSetting

The attributes are self-explanatory but some might only work on specific terminals (colors will work pretty much everywhere). You can have a set of attributes separated by commas. You can get a list of possible attributes on the ansiColor help:

> help ansiColor
-- ansiColor(aAnsi, aString, force) : String
-- -----------------------------------------
-- Returns the ANSI codes together with aString, if determined that the current terminal can handle ANSI codes (overridden by force = true), with the attributes defined in aAnsi. Please use with ansiStart() and ansiStop(). The attributes separated by commas can be:
 
BLACK; RED; GREEN; YELLOW; BLUE; MAGENTA; CYAN; WHITE;
FG_BLACK; FG_RED; FG_GREEN; FG_YELLOW; FG_BLUE; FG_MAGENTA; FG_CYAN; FG_WHITE;
BG_BLACK; BG_RED; BG_GREEN; BG_YELLOW; BG_BLUE; BG_MAGENTA; BG_CYAN; BG_WHITE;
BOLD; FAINT; INTENSITY_BOLD; INTENSITY_FAINT; ITALIC; UNDERLINE; BLINK_SLOW; BLINK_FAST; BLINK_OFF; NEGATIVE_ON; NEGATIVE_OFF; CONCEAL_ON; CONCEAL_OFF; UNDERLINE_DOUBLE; UNDERLINE_OFF;

Examples of usage:

Writing part of a string with white foreground and red background:

Writing part of a string with black foreground and yellow background:

Increasing the history size for openaf-console

The default history size on the openaf-console is around 500 lines or commands. But there are some users that require a little more for their uses. So can an user increase the history size for openaf-console? Yes, you just set it on the openaf-console_profile file.

To check the current size just open an openaf-console and execute:

> con.getConsoleReader().getHistory().getMaxSize()
500

To change this you just need to execute:

con.getConsoleReader().getHistory().setMaxSize(1000);

But it order to make this change permanent you need to execute this setMaxSize every time openaf-console starts. An easy way is to use the openaf-console_profile file. This file, in Unix/Mac, is located at ~/.openaf-console_profile and in Windows its located at c:\users\[you user]\.openaf-console_profile. It executes like an openaf script every time the openaf-console is started.

That means that you have to be careful on what you add to this file as you also have to be careful in increase the history size. The default is set to keep the performance at "satisfactory" levels so do lower your maximum if you notice an non satisfactory performance downgrade.

The same applies to the near by .openaf_profile file. It will get executed every time a openaf script is executed (and keep in mind that the openaf-console is just another openaf script).

Receiving keyboard input

In some scripts you might need to request input from an user or you just want to pause and wait for the user to press any key (the famous "Press any key to continue" sentence). In OpenAF you can do this easily with the Console plugin. Here is a quick example:

plugin("Console");

var con = new Console();
var login = con.readLinePrompt("Please enter your login   : ");
var pass  = con.readLinePrompt("Please enter your password: ", "*");

printnl("Press any key to continue... "); 
print("(you pressed char '" + con.readCharB() + "')\n");

print("Login: " + login);
print("Pass : " + pass);

So the readLinePrompt function will actually read an entire line, with any "prompt" you want and return you what was written. If you need to hide what the user is writing (e.g. password) you can also provide the character to use for that.

The readCharB will wait for the user to hit any key and then return you the ascii code of the charactered hit on the keyboard. You can cycle between readCharB waiting for a specific char, for example. You can also use readChar that let's you pass, as an argument, the allowed characters set. And if just want to check if any character was hit on the keyboard without waiting for it you can use readCharNB.

Format bytes abbreviation

Whenever dealing with bytes you might want to output an abbreviation for human reading like instead of saying 123456789 bytes saying it's around 118MB. You can do this in OpenAF using the ow.format.toBytesAbbreviation function like this:

ow.loadFormat();

var folderPath = ".";
var sumBytes = ow.format.toBytesAbbreviation( 
                 $from( io.listFiles(folderPath).files).sum("size") )
               );

print(sumBytes);

In this example it will sum all the byte sizes of a given folder and print you the corresponding abbreviation in bytes.

Other examples:

> ow.format.toBytesAbbreviation(10)
10 bytes
> ow.format.toBytesAbbreviation(10000)
9.77 KB
> ow.format.toBytesAbbreviation(10000000000000)
9.09 TB

Date diff

Continuing on the javascript Date manipulation functions available in the OpenAF's ow.format library when it comes to comparing date objects. Usually the question between to dates is how many milliseconds, seconds, minutes, hours, days, weeks, months and/or years the date differ. So there is a set of functions, just for date, under ow.format.dateDiff.in*(dateBefore, dateAfter):

ow.loadFormat();

// Get a date last year
var beforeDate = ow.format.toDate("20180503 025401", "yyyyMMdd HHmmss");
var afterDate = ow.format.toDate("20190701 152012", "yyyyMMdd HHmmss");

print(ow.format.dateDiff.inYears(beforeDate, afterDate)); // 1
print(ow.format.dateDiff.inMonths(beforeDate, afterDate)); // 14
print(ow.format.dateDiff.inWeeks(beforeDate, afterDate)); // 60
print(ow.format.dateDiff.inDays(beforeDate, afterDate)); // 424
print(ow.format.dateDiff.inHours(beforeDate, afterDate)); // 10188
print(ow.format.dateDiff.inMinutes(beforeDate, afterDate)); // 611306
print(ow.format.dateDiff.inSeconds(beforeDate, afterDate)); // 36678371
print(afterDate - beforeDate); // in ms, 36678371000

If you don't provide an afterDate it will just default to the current date.