Test latency

The usual latency test most of us use if simply execute the ping operating system command from the source to the target we want to measure. But nowadays, ICMP packets (what ping really transmits) might be blocked by a firewall or similar.

With OpenAF's function ow.format.testLatency that isn't a problem since it will try to open a TCP socket to the desired host and port and measure the time taken for the socket connection to be created (and then it closes it).

It's not perfect but is usually enough to understand the relative latency to another host and port.

First I will present a test function a later show how to use it as an one-liner.

The test function

Here is a test function that will taken several measures and return you: each measure, the average and a chart representation of the variation between each measure.

ow.loadFormat();

var tl = (host, port, times) => { 
    times = _$(times).isNumber().default(3); 
    var tries = [], sum = 0, max = 0; 

    for(var ii = 0; ii < times; ii++) {
        tries.push({
            sample: ii+1, 
            latency: ow.format.testPortLatency(host, port)
        });
    }; 
    tries.forEach((v) => {
        sum += v.latency; 
        max = (v.latency > max) ? v.latency : max; 
        v.chart = ow.format.string.progress(v.latency, max, 0, 50, "=", " ");
    }); 
    tries.push({ 
        sample: "avg", 
        latency: Math.floor(sum/times) + "ms"
    });

    return tries;
}

The one-liner

Now convert it to a one-liner and write on an openaf-console, for example:

> ow.loadFormat();
> var tl = (host, port, times) => { times = _$(times).isNumber().default(3);var tries = [], sum = 0, max = 0; for(var ii = 0; ii < times; ii++) { tries.push({ sample: ii+1, latency: ow.format.testPortLatency(host, port)});}; tries.forEach((v) => { sum += v.latency; max = (v.latency > max) ? v.latency : max; v.chart = ow.format.string.progress(v.latency, max, 0, 50, "=", " "); }); tries.push({ sample: "avg", latency: Math.floor(sum/times) + "ms" }); return tries; }

And test it:

> table tl("www.yahoo.com", 443);
sample|latency|                      chart                       
------+-------+--------------------------------------------------
1     |63     |==================================================
2     |60     |================================================  
3     |63     |==================================================
avg   |62ms   

Of course, the chart would only help you when there are slight or significant variations in latency between tests:

> table tl("dynamodb.ap-southeast-2.amazonaws.com", 443, 15);
sample|latency|                      chart                       
------+-------+--------------------------------------------------
1     |354    |==================================================
2     |364    |==================================================
3     |357    |================================================= 
4     |351    |================================================  
5     |352    |================================================  
6     |357    |================================================= 
7     |406    |==================================================
8     |358    |============================================      
9     |354    |============================================      
10    |348    |===========================================       
11    |361    |============================================      
12    |358    |============================================      
13    |369    |=============================================     
14    |354    |============================================      
15    |366    |=============================================     
avg   |360ms  

From the command-line

If you want to run it from the command-line:

$ openaf -c 'ow.loadFormat();var tl = (host, port, times) => { times = _$(times).isNumber().default(3);var tries = [], sum = 0, max = 0; for(var ii = 0; ii < times; ii++) { tries.push({ sample: ii+1, latency: ow.format.testPortLatency(host, port)});}; tries.forEach((v) => { sum += v.latency; max = (v.latency > max) ? v.latency : max; v.chart = ow.format.string.progress(v.latency, max, 0, 50, "=", " "); }); tries.push({ sample: "avg", latency: Math.floor(sum/times) + "ms" }); return tries; }; print(printTable(tl(  "www.google.com", 443, 15)));'
sample|latency|                      chart                       
------+-------+--------------------------------------------------
1     |35     |==================================================
2     |30     |===========================================       
3     |30     |===========================================       
4     |26     |=====================================             
5     |30     |===========================================       
6     |29     |=========================================         
7     |29     |=========================================         
8     |28     |========================================          
9     |28     |========================================          
10    |29     |=========================================         
11    |33     |===============================================   
12    |30     |===========================================       
13    |32     |==============================================    
14    |33     |===============================================   
15    |33     |===============================================   
avg   |30ms   

And replace the function arguments on the end of the one-liner.

Testing a TCP port

When performing TCP connections you always have to deal with the eventual connection failure. For example, if you write a script that will connect to a specific server, you should deal with the issue that when executing the script might not have connectivity to the desire target.

So how to handle these events? The typical answer is waiting for the connection error exception and deal with it. For example:

try {
    // make the TCP call
} catch(e) {
    // handle the exception
}

You should always handle the exception but you can avoid even going into the try/catch block with a quick TCP connectivy check:

ow.loadFormat();

var host = "my.service.host";
var port = 1234;

if (ow.format.testPort(host, port)) {
    var result;
    try {
        result = callService(host, port, params);
        // process result
    } catch(e) {
        logErr("Problem calling service on " + host + ":" + port);
    }
} else {
    logErr("No connectivity to " + host + ":" + port);
}

The ow.format.testPort function allows for quick socket connection tests. By default it timeouts after 1.5 seconds but you can change that using a third parameter:

// Wait 5 seconds before declaring farAway service not reachable (false)
ow.format.testPort(farAwayHost, farAwayPort, 5000);

Converting numbers

Is there a way, in OpenAF, to convert a decimal number to hex? Or to binary? Or to octal? Yes, there is on the ow.format library.
Start by loading the library:

> ow.loadFormat();

Converting from hexadecimal to decimal:

> ow.format.fromHex("1F78")
8056

Converting from decimal to hexadecimal:

> ow.format.toHex(8056)
"1f78"

Converting from decimal to octal:

> ow.format.toOctal(8056)
"17570"

Converting from octal to decimal:

> ow.format.fromOctal(17570)
8056

Converting from decimal to binary:

> ow.format.toBinary(12345)
"11000000111001"

Converting from binary to decimal:

> ow.format.fromBinary("11000000111001")
12345

And that's it. For conversions between arrays of bytes and the corresponding hexadecimal representation there are also functions like ow.format.string.toHex and ow.format.string.toHexArray.

Format bytes abbreviation

Whenever dealing with bytes you might want to output an abbreviation for human reading like instead of saying 123456789 bytes saying it's around 118MB. You can do this in OpenAF using the ow.format.toBytesAbbreviation function like this:

ow.loadFormat();

var folderPath = ".";
var sumBytes = ow.format.toBytesAbbreviation( 
                 $from( io.listFiles(folderPath).files).sum("size") )
               );

print(sumBytes);

In this example it will sum all the byte sizes of a given folder and print you the corresponding abbreviation in bytes.

Other examples:

> ow.format.toBytesAbbreviation(10)
10 bytes
> ow.format.toBytesAbbreviation(10000)
9.77 KB
> ow.format.toBytesAbbreviation(10000000000000)
9.09 TB

Date diff

Continuing on the javascript Date manipulation functions available in the OpenAF's ow.format library when it comes to comparing date objects. Usually the question between to dates is how many milliseconds, seconds, minutes, hours, days, weeks, months and/or years the date differ. So there is a set of functions, just for date, under ow.format.dateDiff.in*(dateBefore, dateAfter):

ow.loadFormat();

// Get a date last year
var beforeDate = ow.format.toDate("20180503 025401", "yyyyMMdd HHmmss");
var afterDate = ow.format.toDate("20190701 152012", "yyyyMMdd HHmmss");

print(ow.format.dateDiff.inYears(beforeDate, afterDate)); // 1
print(ow.format.dateDiff.inMonths(beforeDate, afterDate)); // 14
print(ow.format.dateDiff.inWeeks(beforeDate, afterDate)); // 60
print(ow.format.dateDiff.inDays(beforeDate, afterDate)); // 424
print(ow.format.dateDiff.inHours(beforeDate, afterDate)); // 10188
print(ow.format.dateDiff.inMinutes(beforeDate, afterDate)); // 611306
print(ow.format.dateDiff.inSeconds(beforeDate, afterDate)); // 36678371
print(afterDate - beforeDate); // in ms, 36678371000

If you don't provide an afterDate it will just default to the current date.

Date to/from string

To help with javascript Date manipulation in OpenAF there are several helper functions in the ow.format library. To use them do load the format library first:

ow.loadFormat();

One of the most used date functions is the ow.format.toDate and the ow.format.fromDate. These helper functions let you convert strings into Dates and Dates into strings.

var someDate = ow.format.toDate("20170504", "yyyyMMdd");
print(someDate); // "2017-05-04T00:00:00.000Z"

var aString = ow.format.fromDate(new Date(), "yyyyMMdd-HHmmss");
print(aString); // "20190730-000000"

They are very similar, in spirit, to the SQL to_date and to_char functions but you have to be careful that they use the Java mask format:

  G - Era descriptor (AD)
  y - Year (1996; 96)
  Y - Week year (2009; 09)
  M - Month in year (July; Jul; 07)
  w - Week in year (27)
  W - Week in month (2)
  D - Day in year (189)
  d - Day in month (10)
  F - Day of week in month (2)
  E - Day name in week (Tuesday; Tue)
  u - Day number of week (1 = Monday, ..., 7 = Sunday) (1)
  a - Am/pm number (PM)
  H - Hour in day (0-23)
  k - Hour in day (1-24)
  K - Hour in am/pm (0-11)
  h - Hour in am/pm (1-12)
  m - Minute in hour (30)
  s - Second in minute (55)
  S - Millisecond (978)
  z - Time zone (Pacific Standard Time; PST; GMT-08:00)
  Z - Time zone (-0800)
  X - Time zone (-08; -0800; -08:00)

Adding an array to an Excel spreadsheet

The OpenAF's XLS plugin offers one of the most handy features: the ability to write a simple javascript array of maps to an Excel spreadsheet. But first a warning: it can't be a complex sub maps/arrays, just plain strings/numbers array which usually is enough (although there is an alternative way that will mention on the end of the post).

The functionality is captured on the setTable function of the XLS plugin. Giving an example, let's say we get an array with all the files and corresponding info from a folder using io.listFiles:

var path = ".";
var outputFile = "test.xlsx";

var listOfFiles = io.listFiles(path).files; 
// files is an array returned by io.listFiles with filesystem details of files & folders on the provided path

plugin("XLS");
var xls = new XLS(); 

// Determines in which sheet the array will be added
var sheet = xls.getSheet("my sheet"); 

// Writes all the array elements and corresponding properties to the provided sheet starting on excel position B2.
xls.setTable(sheet, "B", "2", listOfFiles); 

// Writes the prepared excel to a xls/xlsx file.
xls.writeFile(outputFile);
xls.close(); 
// Don't forget to close the object to free up used files and resources before using the generated excel file.

On the first lines of the code we defined the output file as "test.xlsx". After running this script:

openaf -f test.js

if there wasn't any errors you will find a test.xlsx on the same folder that will look similar to this:

The first instinct is: "can I format it?" The answer is yes. You can add an auto-filter easily:

ow.loadFormat();
ow.format.xls.autoFilter(sheet, "B2:K2")

"Can I change color, font, etc...?": Yes, check out ow.format.xls.getStyle.

"Can I just use a previous excel template and just fill it in?": Yes. The probably the easiest to do. Just change the new XLS line to this:

var xls = new XLS("myTemplate.xlsx");