Scripting

From DreamFactory
Jump to: navigation, search

DreamFactory supports server-side scripting to quickly and easily customize almost all of the platform's REST API endpoints to include business logic on the server, such as field validation, workflow triggers, runtime calculations, and more. Developers can easily attach scripts to any existing API endpoint, for both pre- and post-processing of the request and response. You can also write your own custom REST APIs with server-side script services. Starting with release 2.3.0, scripts can even be queued for later processing. This section provides an overview of how scripting works. Please see the tutorials section for example scripts.

Supported Scripting Languages

DreamFactory scripting supports several modern scripting languages. To get a list of which ones are installed and setup on a particular instance use the following API. 

http:/example.com/api/v2/system/script_type
{
  "resource": [
    {
      "name": "nodejs",
      "label": "Node.js",
      "description": "Server-side JavaScript handler using the Node.js engine.",
      "sandboxed": false
    },
    {
      "name": "php",
      "label": "PHP",
      "description": "Script handler using native PHP.",
      "sandboxed": false
    },
    {
      "name": "python",
      "label": "Python",
      "description": "Script handler using native Python.",
      "sandboxed": false
    },
    {
      "name": "v8js",
      "label": "V8js",
      "description": "Server-side JavaScript handler using the V8js engine.",
      "sandboxed": true
    }
  ]
}

Note: The sandbox setting means that the script execution is bound by memory and time and is not allowed access to other operating system functionalities outside of PHP's context. This is currently only the case for V8Js. Therefore, be aware that DreamFactory cannot control what is done inside scripts using non-sandboxed languages on a server.

The following are typically supported on most installs:

Where Scripting Can Be Used

Server-side scripts can be used in two areas of a DreamFactory instance. One of which is attached to system events which may be triggered by internal events, API calls, or other scripts. For more on events and how to apply scripting to them, go to Event Scripting.

The other way to use server-side scripts in DreamFactory is to use customizable script services. There is a scripting service type for each supported scripting language. For more on using scripting as a service, go to Script Services.

Resources Available To A Script

When a script is executed, DreamFactory passes in two very useful resources that allow each script to access many parts of the system including system states, configuration, and even a means to call other services or external APIs. They are the event resource and the platform resource.

Note: The term "resource" is used generically here, based on the scripting language used, the resource could either be an object (i.e. V8js or Node.js) or an array (i.e. PHP).

The Event Resource

The event resource contains the structured data about the event triggered (Event Scripting) or from the API service call (Script Services). As seen below, this includes things like the request and response information available to this "event".

Note: Determined by the type of event triggering the script, parts of this event resource are writable. Modifications to this resource while executing the script do not result in a change to that resource (i.e. request or response) in further internal handling of the API call, unless the event script is configured with the allow_event_modification setting to true, or it is the response on a script service. Prior to 2.1.2, the allow_event_modification was accomplished by setting a content_changed element in the request or response object to true.

The event resource has the following properties:

Property Type Description
request resource A resource representing the inbound REST API call, i.e. the HTTP request.
response resource A resource representing the response to an inbound REST API call, i.e. the HTTP response.
resource string Any additional resource names typically represented as a replaceable part of the path, i.e. "table name" on a db/_table/{table_name} call.

Event Request

The "request" resource contains all the components of the original HTTP request. This resource is always available, and is writable during pre-process event scripting.

Property Type Description
api_version string The API version used for the request (i.e. 2.0).
method string The HTTP method of the request (i.e. GET, POST, PUT).
parameters resource An object/array of query string parameters received with the request, indexed by the parameter name.
headers resource An object/array of HTTP headers from the request, indexed by the lowercase header name.
content string The body of the request in raw string format.
content_type string The format type (i.e. "application/json") of the raw content of the request.
payload resource The body (POST body) of the request, i.e. the content, converted to an internally usable object/array if possible.

Any allowed changes to this data will overwrite existing data in the request, before further listeners are called and/or the request is handled by the called service.

Event Response

The response resource contains the data being sent back to the client from the request.

Note: This resource is only available/relevant on post-process event and script service scripts.

Property Type Description
status_code integer The HTTP status code of the response (i.e. 200, 404, 500, etc).
headers resource An object/array of HTTP headers for the response back to the client.
content mixed The body of the request as an object if the content_type is not set, or in raw string format.
content_type string The content type (i.e. json) of the raw content of the request.

Just like request, any allowed changes to response will overwrite existing data in the response, before it is sent back to the caller.

The Platform Resource

This platform resource may be used to access configuration and system states, as well as, the REST API of your instance via inline calls. This makes internal requests to other services directly without requiring an HTTP call.

The platform resource has the following properties:

Field Type Description
api resource An array/object that allows access to the instance's REST API.
config resource An array/object consisting of the current configuration of the instance.
session resource An array/object consisting of the current session information.

Platform API

The api resource contains methods for instance API access. This object contains a method for each type of REST verb.

Function Description
get GET a resource
post POST a resource
put PUT a resource
patch PATCH a resource
delete DELETE a resource

They all accept the same arguments: 

method( "service[/resource_path]"[, payload[, options]] );
  • method - Required. The method/verb listed above.
  • service - Required. The service name (as used in API calls) or external URI.
  • resource_path - Optional depending on your call. Resources of the service called.
  • payload - Optional, but must contain a valid object for the language of the script.
  • options - Optional, may contain headers, query parameters, and cURL options.

Calling internally only requires the relative URL without the /api/v2/ portion. You can pass absolute URLs like 'http://example.com/my_api' to these methods to access external resources. See the scripting tutorials for more examples of calling platform.api methods from scripts. 

// V8js
 
var url = 'db/_table/contact';
var result = platform.api.get(url);
var_dump(result);
// Node.js
 
var url = 'db/_table/contact';
var options = null;
platform.api.get(url, options, function(body, response) {
        var result = JSON.parse(body);
        console.log(result);
});
// PHP

$url = 'db/_table/contact';
$api = $platform['api'];
$get = $api->get;
$result = $get($url);
var_dump($result);

// Python
 
url = 'db/_table/contact'
result = platform.api.get(url)
data = result.read()
print data
jsonData = bunchify(json.loads(data))

Modifying Request Parameters

event.request.parameters.limit = 2

Retrieving a Request Parameter

To retrieve a request parameter using PHP, you'll reference it the parameter name via the $event['request']['parameters'] associative array: 

// PHP
$customerKey = $event['request']['parameters']['customer_key'];

To retrieve the filter parameter, reference the filter key: 

// PHP
$filter = $event['request']['parameters']['filter']

This will return the key/value pair, such as "id=50". Therefore you'll want to use a string parsing function such as PHP's explode() to retrieve the key value: 

// PHP
$id = explode("=", $event['request']['parameters']['filter'])[1];

To retrieve a header value: 

# Python
request = event.request
print request.headers['x-dreamfactory-api-key']

Throwing an Exception

If a parameter such as filter is missing, can throw an exception like so: 

// PHP
if (! array_key_exists('filter', $event['request']['parameters'])) {
    throw new \DreamFactory\Core\Exceptions\BadRequestException('Missing filter');
}

Adding HTTP headers, query parameters, or cURL options to api calls

You can specify any combination of headers and query parameters when calling platform.api functions from a script. This is supported by all script types using the options argument. 

// V8js
 
var url = 'http://example.com/my_api';
var payload = {"name":"test"};
var options = {
    'headers': {
        'Content-Type': 'application/json'
    },
    'parameters': {
        'api_key': 'my_api_key'
    },
};
var result = platform.api.post(url, payload, options);
var_dump(result);
// Node.js
 
var url = 'http://example.com/my_api';
var payload = {"name":"test"};
var options = {
    'headers': {
        'Content-Type': 'application/json'
    },
    'parameters': {
        'api_key': 'my_api_key'
    },
};
platform.api.post(url, payload, options, function(body, response) {
        var result = JSON.parse(body);
        console.log(result);
}
// PHP

$url = 'http://example.com/my_api';
$payload = json_decode("{\"name\":\"test\"}", true);
$options = [];
$options['headers'] = [];
$options['headers']['Content-Type'] = 'application/json';
$options['parameters'] = [];
$options['parameters']['api_key'] = 'my_api_key';
$api = $platform['api'];
$post = $api->post;
$result = $post($url, $payload, $options);
var_dump($result);

// Python
 
url = 'http://example.com/my_api'
payload = '{\"name\":\"test\"}'
options = {}
options['headers'] = {}
options['headers']['Content-Type'] = 'application/json'
options['parameters'] = {}
options['parameters']['api_key'] = 'my_api_key'
result = platform.api.post(url, payload, options)
data = result.read()
print data
jsonData = bunchify(json.loads(data))

For V8js and PHP scripts, which use cURL to make calls to external URLs, you can also specify any number of cURL options. Calls to internal URLs do not use cURL, so cURL options have no effect there. 

// V8js
 
options = {
    'headers': {
        'Content-Type': 'application/json'
    },
    'CURLOPT_USERNAME' : '[email protected]'
    'CURLOPT_PASSWORD' : 'password123'
};
// PHP

$options = [];
$options['headers'] = [];
$options['headers']['Content-Type'] = 'application/json';
$options['parameters'] = [];
$options['parameters']['api_key'] = 'my_api_key';
$options['CURLOPT_USERNAME'] = '[email protected]';
$options['CURLOPT_PASSWORD'] = 'password123';

cURL options can include HTTP headers using CURLOPT_HTTPHEADER, but it's recommended to use options.headers for V8js or $options['headers'] for PHP to send headers as shown above.

Platform Config

The config object contains configuration settings for the instance.

Function Description
df Configuration settings specific to DreamFactory. 
       (
           [version] => "2.1.0"
           [api_version] => "2.0"
           [always_wrap_resources] => true
           [resources_wrapper] => "resource"
           [storage_path] => "my/install/path/storage"
           ...
       )

Platform Session

The session resource contains information and states about the current session for the event.

Function Description
api_key DreamFactory API key.
session_token Session token, i.e. JWT.
user User information derived from the supplied session token, i.e. JWT. 
       (
           [id] => 6
           [display_name] => First Last
           [first_name] => First
           [last_name] => Last
           [email] => [email protected]
           [is_sys_admin] => 1
           [last_login_date] => 2016-02-19 14:05:25
       )
app App information derived from the supplied API key.
lookup Available lookups for the session.

Stopping Script Execution

Just like in normal code execution, execution of a script is stopped prematurely by two means, throwing an exception, or returning. 

// Stop execution if verbs other than GET are used in Custom Scripting Service
if (event.request.method !== "GET") {
    throw "Only HTTP GET is allowed on this endpoint."; // will result in a 500 back to client with the given message.
}
 
// Stop execution and return a specific status code
if (event.resource !== "test") {
    // For pre-process scripts where event.response doesn't exist yet, just create it
    event.response = {};
    // For post-process scripts just update the members necessary
    event.response.status_code = 400;
    event.response.content = {"error": "Invalid resource requested."};
    return;
}
 
// defaults to 200 status code
event.response.content = {"test": "value"};

Queued Scripting Setup

DreamFactory queued scripting takes advantage of Laravel's built-in queueing feature, for more detailed information, see their documentation here. Every DreamFactory instance comes already setup with the 'database' queue setting with all necessary tables created (scripts and failed_scripts). The queue configuration file is stored in config/queue.php and can be updated if another setup is preferred, such as Beanstalkd, Amazon SQS, or Redis.

DreamFactory also fully supports the following artisan commands for configuration and runtime execution... 

 queue:failed                       List all of the failed queue scripts
 queue:flush                        Flush all of the failed queue scripts
 queue:forget                       Delete a failed queue script
 queue:listen                       Listen to a given queue
 queue:restart                      Restart queue worker daemons after their current script
 queue:retry                        Retry a failed queue script
 queue:work                         Process the next script on a queue

Specifying The Queue

You may also specify the queue a script should be sent to. By pushing scripts to different queues, you may categorize your queued scripts, and even prioritize how many workers you assign to various queues. This does not push scripts to different queue connections as defined by your queue configuration file, but only to specific queues within a single connection. To specify the queue, use the queue configuration option on the script or service.

Specifying The Queue Connection

If you are working with multiple queue connections, you may specify which connection to push a script to. To specify the connection, use the connection configuration option on the script or service.

Delayed Scripts

Sometimes you may wish to delay the execution of a queued script for some period of time. For instance, you may wish to queue a script that sends a customer a reminder e-mail 5 minutes after sign-up. You may accomplish this using the delay configuration option on your script or service. The option values should be in seconds.

Running The Queue Listener

Starting The Queue Listener

Laravel includes an Artisan command that will run new scripts as they are pushed onto the queue. You may run the listener using the queue:listen command: 

php artisan queue:listen

You may also specify which queue connection the listener should utilize: 

php artisan queue:listen connection-name

Note that once this task has started, it will continue to run until it is manually stopped. You may use a process monitor such as <a href="http://supervisord.org/">Supervisor</a> to ensure that the queue listener does not stop running.

Queue Priorities

You may pass a comma-delimited list of queue connections to the listen script to set queue priorities: 

php artisan queue:listen --queue=high,low

In this example, scripts on the high queue will always be processed before moving onto scripts from the low queue.

Specifying The Script Timeout Parameter

You may also set the length of time (in seconds) each script should be allowed to run: 

php artisan queue:listen --timeout=60

Specifying Queue Sleep Duration

In addition, you may specify the number of seconds to wait before polling for new scripts: 

php artisan queue:listen --sleep=5

Note that the queue only sleeps if no scripts are on the queue. If more scripts are available, the queue will continue to work them without sleeping.

Processing The First Script On The Queue

To process only the first script on the queue, you may use the queue:work command: 

php artisan queue:work

Dealing With Failed Scripts

To specify the maximum number of times a script should be attempted, you may use the --tries switch on the queue:listen command: 

php artisan queue:listen connection-name --tries=3

After a script has exceeded this amount of attempts, it will be inserted into a failed_jobs table.

Retrying Failed Scripts

To view all of your failed scripts that have been inserted into your failed_jobs database table, you may use the queue:failed Artisan command: 

php artisan queue:failed

The queue:failed command will list the script ID, connection, queue, and failure time. The script ID may be used to retry the failed script. For instance, to retry a failed script that has an ID of 5, the following command should be issued: 

php artisan queue:retry 5

To retry all of your failed scripts, use queue:retry with all as the ID: 

php artisan queue:retry all

If you would like to delete a failed script, you may use the queue:forget command: 

php artisan queue:forget 5

To delete all of your failed scripts, you may use the queue:flush command: 

php artisan queue:flush

DreamFactory Scripting Library

Obtain DreamFactory API Key Details

A simple PHP script to obtain additional information regarding the API Key used such as the Keys name, active status, and more. 

$app_id = $platform['session']['app']['id'];
$url = 'system/app?ids=' . $app_id;
$api = $platform['api'];
$get = $api->get;
$result = $get($url);

return $result;


Use DreamFactory's scripted service to add a Database function

A DreamFactory feature that allows the administrator to add a database function to a column so when that column is retrieved by the API, the function runs in its place.For instance, imagine if you want to change the format of the date field, you could use ORACLE’s TO_DATE() function to do that:

TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM') 

$api = $platform['api'];
$get = $api->get;
$patch = $api->patch;
$options = [];
set_time_limit(800000);
// Get all tables URL. Replace the databaseservicename with your API namespace
$url = '<API_Namespace>/_table';

// Call parent API
$result = $get($url);
$fieldCount = 0;
$tableCount = 0;
$tablesNumber = 0;

// Check status code
if ($result['status_code'] == 200) {
// If parent API call returns 200, call a MySQL API
    $tablesNumber = count($result['content']['resource']);

// The next line is to limit number of tables to first 5 to see the successfull run of the script
//$result['content']['resource'] = array_slice($result['content']['resource'], 0, 5, true);

foreach ($result['content']['resource'] as $table) {
    // Get all fields URL
    $url = "<API_Namespace>/_schema/" . $table['name'] . "?refresh=true";
    $result = $get($url);
     
    if ($result['status_code'] == 200) {
        $tableCount++;
        foreach ($result['content']['field'] as $field) {
            if (strpos($field['db_type'], 'date') !== false || strpos($field['db_type'], 'Date') !== false || strpos($field['db_type'], 'DATE') !== false) {
                // Patch field URL
                $fieldCount++;
                $url = "<API_Namespace>/_schema/" . $table['name'] . "/_field";
                
                // Skip fields that already have the function
                if ($field['db_function'][0]['function'] === "TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM')") continue;
                // Remove broken function
                $field['db_function'] = null;
                $payload = ['resource' => [$field]];
                $result = $patch($url, $payload);
                
                // Add correct function
                $field['db_function'] = [['function' => "TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM')", "use" => ["INSERT", "UPDATE"]]];
                $payload = ['resource' => [$field]];
                $result = $patch($url, $payload);
                
                if ($result['status_code'] == 200) {
                    echo("Function successfully added to " . $field['label'] . " field in " . $table['name'] . " table \n");
                    \Log::debug("Function successfully added to " . $field['label'] . " field in " . $table['name'] . " table");

                } else {
                    $event['response'] = [
                        'status_code' => 500,
                        'content' => [
                            'success' => false,
                            'message' => "Could not add function to " . $field['label'] . " in " . $table['name'] . " table;"
                        ]
                    ];

                }
            } 
        }
        \Log::debug("SCRIPT DEBUG: Total tables number " . $tablesNumber . " -> Tables  " . $tableCount . " fieldCount " . $fieldCount);
    } else {
        $event['response'] = [
            'status_code' => 500,
            'content' => [
                'success' => false,
                'message' => "Could not get all fields."
            ]
        ];
    }
}
} else {
$event['response'] = [
    'status_code' => 500,
    'content' => [
        'success' => false,
        'message' => "Could not get list of tables."
    ]
];
}
return "Script finished";


Email Alert API Usages

If there is a suspicious amount of API usage, send an email notification to an admin. This is a custom scripted service that uses a 'TransactionHistory' table to count the number of API calls in the last 5 minutes. An email alert will be sent to the specified email address if the number of API calls exceeds a specific threshold. For this example we are monitoring the usage of the API for the past 5 minutes and setting a threshold of 1,000 API calls. To modify these values to fit your project just change the 'timelapse' and 'threshold' values.

Note: This script can also be scheduled to run periodically using DreamFactory's built in Scheduling feature. 

function twoDigits(d) {
    if (0 <= d && d < 10) return "0" + d.toString();
    if (-10 < d && d < 0) return "-0" + (-1 * d).toString();
    return d.toString();
}

var toMysqlFormat = function (x) {
    return x.getUTCFullYear() + "-" + twoDigits(1 + x.getUTCMonth()) + "-" + twoDigits(x.getUTCDate()) + " " + twoDigits(x.getUTCHours()) + ":" + twoDigits(x.getUTCMinutes()) + ":" + twoDigits(x.getUTCSeconds());
};

var timelapse = 5; // How many minutes to count

var threshold = 1000; // How many API calls are allowed

var query = "timestamp%3c" + toMysqlFormat(new Date(Date.now() - timelapse * 60000));

result = platform.api.get("db/_table/TransactionHistory?filter=(" + query + ")");

if (result.content.resource.length >= threshold) {

    var email = {

        "to": [
            {
                "name": "John Doe",
                "email": "[email protected]"
            }
        ],
        "subject": "Usage alert",
        "body_text": "API usage has exceeded the alert threshold.",
        "from_name": "John Doe",
        "from_email": "[email protected]",
        "reply_to_name": "John Doe",
        "reply_to_email": "[email protected]"
    };

    platform.api.post("email", email);
}

Create Custom Error Messages

You may wish to customize error messages if you are validating input parameters or simply want to guide the end user with a fleshed out response. To achieve this you can add a pre-process event handler with the code below. 

// PHP
throw new \Exception('Custom Message');

// Python
raise ValueError('Custom Message')

// NodeJS
throw "Custom Message";
Retrieved from "https://wiki.dreamfactory.com/index.php?title=DreamFactory/Features/Scripting&oldid=34949"