Scripting
DreamFactory supports server-side scripting to quickly and easily customize almost all of the platform's REST API endpoints to include business logic on the server, such as field validation, workflow triggers, runtime calculations, and more. Developers can easily attach scripts to any existing API endpoint, for both pre- and post-processing of the request and response. You can also write your own custom REST APIs with server-side script services. Starting with release 2.3.0, scripts can even be queued for later processing. This section provides an overview of how scripting works. Please see the tutorials section for example scripts.
Contents
- 1 Supported Scripting Languages
- 2 Where Scripting Can Be Used
- 3 Resources Available To A Script
- 4 Stopping Script Execution
- 5 Queued Scripting Setup
- 6 DreamFactory Scripting Library
- 6.1 DreamFactory Scripting Library
- 6.2 Obtain DreamFactory API Key Details
- 6.3 Use DreamFactory's scripted service to add a Database function
- 6.4 Email Alert API Usages
- 6.5 Create Custom Error Messages
- 6.6 NodeJS script to POST records to a Database
- 6.7 Validating API Payloads in Python
- 6.8 Python API Proxy
- 6.9 What is CORS?
- 6.10 Call a Stored Procedure From REST API
Supported Scripting Languages
DreamFactory scripting supports several modern scripting languages. To get a list of which ones are installed and setup on a particular instance use the following API.
http:/example.com/api/v2/system/script_type
{ "resource": [ { "name": "nodejs", "label": "Node.js", "description": "Server-side JavaScript handler using the Node.js engine.", "sandboxed": false }, { "name": "php", "label": "PHP", "description": "Script handler using native PHP.", "sandboxed": false }, { "name": "python", "label": "Python", "description": "Script handler using native Python.", "sandboxed": false }, { "name": "v8js", "label": "V8js", "description": "Server-side JavaScript handler using the V8js engine.", "sandboxed": true } ] }
Note: The sandbox setting means that the script execution is bound by memory and time and is not allowed access to other operating system functionalities outside of PHP's context. This is currently only the case for V8Js. Therefore, be aware that DreamFactory cannot control what is done inside scripts using non-sandboxed languages on a server.
The following are typically supported on most installs:
Where Scripting Can Be Used
Server-side scripts can be used in two areas of a DreamFactory instance. One of which is attached to system events which may be triggered by internal events, API calls, or other scripts. For more on events and how to apply scripting to them, go to Event Scripting.
The other way to use server-side scripts in DreamFactory is to use customizable script services. There is a scripting service type for each supported scripting language. For more on using scripting as a service, go to Script Services.
Resources Available To A Script
When a script is executed, DreamFactory passes in two very useful resources that allow each script to access many parts of the system including system states, configuration, and even a means to call other services or external APIs. They are the event resource and the platform resource.
Note: The term "resource" is used generically here, based on the scripting language used, the resource could either be an object (i.e. V8js or Node.js) or an array (i.e. PHP).
The Event Resource
The event resource contains the structured data about the event triggered (Event Scripting) or from the API service call (Script Services). As seen below, this includes things like the request and response information available to this "event".
Note: Determined by the type of event triggering the script, parts of this event resource are writable. Modifications to this resource while executing the script do not result in a change to that resource (i.e. request or response) in further internal handling of the API call, unless the event script is configured with the allow_event_modification setting to true, or it is the response on a script service. Prior to 2.1.2, the allow_event_modification was accomplished by setting a content_changed element in the request or response object to true.
The event resource has the following properties:
Property | Type | Description |
---|---|---|
request | resource | A resource representing the inbound REST API call, i.e. the HTTP request. |
response | resource | A resource representing the response to an inbound REST API call, i.e. the HTTP response. |
resource | string | Any additional resource names typically represented as a replaceable part of the path, i.e. "table name" on a db/_table/{table_name} call. |
Event Request
The "request" resource contains all the components of the original HTTP request. This resource is always available, and is writable during pre-process event scripting.
Property | Type | Description |
---|---|---|
api_version | string | The API version used for the request (i.e. 2.0). |
method | string | The HTTP method of the request (i.e. GET, POST, PUT). |
parameters | resource | An object/array of query string parameters received with the request, indexed by the parameter name. |
headers | resource | An object/array of HTTP headers from the request, indexed by the lowercase header name. |
content | string | The body of the request in raw string format. |
content_type | string | The format type (i.e. "application/json") of the raw content of the request. |
payload | resource | The body (POST body) of the request, i.e. the content, converted to an internally usable object/array if possible. |
Any allowed changes to this data will overwrite existing data in the request, before further listeners are called and/or the request is handled by the called service.
Event Response
The response resource contains the data being sent back to the client from the request.
Note: This resource is only available/relevant on post-process event and script service scripts.
Property | Type | Description |
---|---|---|
status_code | integer | The HTTP status code of the response (i.e. 200, 404, 500, etc). |
headers | resource | An object/array of HTTP headers for the response back to the client. |
content | mixed | The body of the request as an object if the content_type is not set, or in raw string format. |
content_type | string | The content type (i.e. json) of the raw content of the request. |
Just like request, any allowed changes to response will overwrite existing data in the response, before it is sent back to the caller.
The Platform Resource
This platform resource may be used to access configuration and system states, as well as, the REST API of your instance via inline calls. This makes internal requests to other services directly without requiring an HTTP call.
The platform resource has the following properties:
Field | Type | Description |
---|---|---|
api | resource | An array/object that allows access to the instance's REST API. |
config | resource | An array/object consisting of the current configuration of the instance. |
session | resource | An array/object consisting of the current session information. |
Platform API
The api resource contains methods for instance API access. This object contains a method for each type of REST verb.
Function | Description |
---|---|
get | GET a resource |
post | POST a resource |
put | PUT a resource |
patch | PATCH a resource |
delete | DELETE a resource |
They all accept the same arguments:
method( "service[/resource_path]"[, payload[, options]] );
- method - Required. The method/verb listed above.
- service - Required. The service name (as used in API calls) or external URI.
- resource_path - Optional depending on your call. Resources of the service called.
- payload - Optional, but must contain a valid object for the language of the script.
- options - Optional, may contain headers, query parameters, and cURL options.
Calling internally only requires the relative URL without the /api/v2/ portion. You can pass absolute URLs like 'http://example.com/my_api' to these methods to access external resources. See the scripting tutorials for more examples of calling platform.api methods from scripts.
// V8js var url = 'db/_table/contact'; var result = platform.api.get(url); var_dump(result);
// Node.js var url = 'db/_table/contact'; var options = null; platform.api.get(url, options, function(body, response) { var result = JSON.parse(body); console.log(result); });
// PHP $url = 'db/_table/contact'; $api = $platform['api']; $get = $api->get; $result = $get($url); var_dump($result);
// Python url = 'db/_table/contact' result = platform.api.get(url) data = result.read() print data jsonData = bunchify(json.loads(data))
Modifying Request Parameters
event.request.parameters.limit = 2
Retrieving a Request Parameter
To retrieve a request parameter using PHP, you'll reference it the parameter name via the $event['request']['parameters'] associative array:
// PHP $customerKey = $event['request']['parameters']['customer_key'];
To retrieve the filter parameter, reference the filter key:
// PHP $filter = $event['request']['parameters']['filter']
This will return the key/value pair, such as "id=50". Therefore you'll want to use a string parsing function such as PHP's explode() to retrieve the key value:
// PHP $id = explode("=", $event['request']['parameters']['filter'])[1];
To retrieve a header value:
# Python request = event.request print request.headers['x-dreamfactory-api-key']
Throwing an Exception
If a parameter such as filter is missing, can throw an exception like so:
// PHP if (! array_key_exists('filter', $event['request']['parameters'])) { throw new \DreamFactory\Core\Exceptions\BadRequestException('Missing filter'); }
Adding HTTP headers, query parameters, or cURL options to api calls
You can specify any combination of headers and query parameters when calling platform.api functions from a script. This is supported by all script types using the options argument.
// V8js var url = 'http://example.com/my_api'; var payload = {"name":"test"}; var options = { 'headers': { 'Content-Type': 'application/json' }, 'parameters': { 'api_key': 'my_api_key' }, }; var result = platform.api.post(url, payload, options); var_dump(result);
// Node.js var url = 'http://example.com/my_api'; var payload = {"name":"test"}; var options = { 'headers': { 'Content-Type': 'application/json' }, 'parameters': { 'api_key': 'my_api_key' }, }; platform.api.post(url, payload, options, function(body, response) { var result = JSON.parse(body); console.log(result); }
// PHP $url = 'http://example.com/my_api'; $payload = json_decode("{\"name\":\"test\"}", true); $options = []; $options['headers'] = []; $options['headers']['Content-Type'] = 'application/json'; $options['parameters'] = []; $options['parameters']['api_key'] = 'my_api_key'; $api = $platform['api']; $post = $api->post; $result = $post($url, $payload, $options); var_dump($result);
// Python url = 'http://example.com/my_api' payload = '{\"name\":\"test\"}' options = {} options['headers'] = {} options['headers']['Content-Type'] = 'application/json' options['parameters'] = {} options['parameters']['api_key'] = 'my_api_key' result = platform.api.post(url, payload, options) data = result.read() print data jsonData = bunchify(json.loads(data))
For V8js and PHP scripts, which use cURL to make calls to external URLs, you can also specify any number of cURL options. Calls to internal URLs do not use cURL, so cURL options have no effect there.
// V8js options = { 'headers': { 'Content-Type': 'application/json' }, 'CURLOPT_USERNAME' : '[email protected]' 'CURLOPT_PASSWORD' : 'password123' };
// PHP $options = []; $options['headers'] = []; $options['headers']['Content-Type'] = 'application/json'; $options['parameters'] = []; $options['parameters']['api_key'] = 'my_api_key'; $options['CURLOPT_USERNAME'] = '[email protected]'; $options['CURLOPT_PASSWORD'] = 'password123';
cURL options can include HTTP headers using CURLOPT_HTTPHEADER, but it's recommended to use options.headers for V8js or $options['headers'] for PHP to send headers as shown above.
Platform Config
The config object contains configuration settings for the instance.
Function | Description |
---|---|
df | Configuration settings specific to DreamFactory.
( [version] => "2.1.0" [api_version] => "2.0" [always_wrap_resources] => true [resources_wrapper] => "resource" [storage_path] => "my/install/path/storage" ... ) |
Platform Session
The session resource contains information and states about the current session for the event.
Function | Description |
---|---|
api_key | DreamFactory API key. |
session_token | Session token, i.e. JWT. |
user | User information derived from the supplied session token, i.e. JWT.
( [id] => 6 [display_name] => First Last [first_name] => First [last_name] => Last [email] => [email protected] [is_sys_admin] => 1 [last_login_date] => 2016-02-19 14:05:25 ) |
app | App information derived from the supplied API key. |
lookup | Available lookups for the session. |
Stopping Script Execution
Just like in normal code execution, execution of a script is stopped prematurely by two means, throwing an exception, or returning.
// Stop execution if verbs other than GET are used in Custom Scripting Service if (event.request.method !== "GET") { throw "Only HTTP GET is allowed on this endpoint."; // will result in a 500 back to client with the given message. } // Stop execution and return a specific status code if (event.resource !== "test") { // For pre-process scripts where event.response doesn't exist yet, just create it event.response = {}; // For post-process scripts just update the members necessary event.response.status_code = 400; event.response.content = {"error": "Invalid resource requested."}; return; } // defaults to 200 status code event.response.content = {"test": "value"};
Queued Scripting Setup
DreamFactory queued scripting takes advantage of Laravel's built-in queueing feature, for more detailed information,
see their documentation here. Every DreamFactory instance comes already setup
with the 'database' queue setting with all necessary tables created (scripts and failed_scripts). The queue
configuration file is stored in config/queue.php
and can be updated if another setup is preferred, such as
Beanstalkd, Amazon SQS, or Redis.
DreamFactory also fully supports the following artisan commands for configuration and runtime execution...
queue:failed List all of the failed queue scripts queue:flush Flush all of the failed queue scripts queue:forget Delete a failed queue script queue:listen Listen to a given queue queue:restart Restart queue worker daemons after their current script queue:retry Retry a failed queue script queue:work Process the next script on a queue
Specifying The Queue
You may also specify the queue a script should be sent to. By pushing scripts to different queues, you may
categorize your queued scripts, and even prioritize how many workers you assign to various queues.
This does not push scripts to different queue connections as defined by your queue configuration file, but
only to specific queues within a single connection. To specify the queue, use the queue
configuration
option on the script or service.
Specifying The Queue Connection
If you are working with multiple queue connections, you may specify which connection to push a script to. To specify
the connection, use the connection
configuration option on the script or service.
Delayed Scripts
Sometimes you may wish to delay the execution of a queued script for some period of time. For instance, you may wish to
queue a script that sends a customer a reminder e-mail 5 minutes after sign-up. You may accomplish this using the
delay
configuration option on your script or service. The option values should be in seconds.
Running The Queue Listener
Starting The Queue Listener
Laravel includes an Artisan command that will run new scripts as they are pushed onto the queue. You may run the
listener using the queue:listen
command:
php artisan queue:listen
You may also specify which queue connection the listener should utilize:
php artisan queue:listen connection-name
Note that once this task has started, it will continue to run until it is manually stopped. You may use a process monitor such as <a href="http://supervisord.org/">Supervisor</a> to ensure that the queue listener does not stop running.
Queue Priorities
You may pass a comma-delimited list of queue connections to the listen
script to set queue priorities:
php artisan queue:listen --queue=high,low
In this example, scripts on the high
queue will always be processed before moving onto scripts from the
low
queue.
Specifying The Script Timeout Parameter
You may also set the length of time (in seconds) each script should be allowed to run:
php artisan queue:listen --timeout=60
Specifying Queue Sleep Duration
In addition, you may specify the number of seconds to wait before polling for new scripts:
php artisan queue:listen --sleep=5
Note that the queue only sleeps if no scripts are on the queue. If more scripts are available, the queue will continue to work them without sleeping.
Processing The First Script On The Queue
To process only the first script on the queue, you may use the queue:work
command:
php artisan queue:work
Dealing With Failed Scripts
To specify the maximum number of times a script should be attempted, you may use the --tries
switch on the
queue:listen
command:
php artisan queue:listen connection-name --tries=3
After a script has exceeded this amount of attempts, it will be inserted into a failed_jobs
table.
Retrying Failed Scripts
To view all of your failed scripts that have been inserted into your failed_jobs
database table, you
may use the queue:failed
Artisan command:
php artisan queue:failed
The queue:failed
command will list the script ID, connection, queue, and failure time. The script ID may
be used to retry the failed script. For instance, to retry a failed script that has an ID of 5, the following command
should be issued:
php artisan queue:retry 5
To retry all of your failed scripts, use queue:retry
with all
as the ID:
php artisan queue:retry all
If you would like to delete a failed script, you may use the queue:forget
command:
php artisan queue:forget 5
To delete all of your failed scripts, you may use the queue:flush
command:
php artisan queue:flush
DreamFactory Scripting Library
DreamFactory Scripting Library
Obtain DreamFactory API Key Details
A simple PHP script to obtain additional information regarding the API Key used such as the Keys name, active status, and more.
$app_id = $platform['session']['app']['id']; $url = 'system/app?ids=' . $app_id; $api = $platform['api']; $get = $api->get; $result = $get($url); return $result;
Use DreamFactory's scripted service to add a Database function
A DreamFactory feature that allows the administrator to add a database function to a column so when that column is retrieved by the API, the function runs in its place.For instance, imagine if you want to change the format of the date field, you could use ORACLE’s TO_DATE() function to do that:
TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM')
$api = $platform['api']; $get = $api->get; $patch = $api->patch; $options = []; set_time_limit(800000); // Get all tables URL. Replace the databaseservicename with your API namespace $url = '<API_Namespace>/_table'; // Call parent API $result = $get($url); $fieldCount = 0; $tableCount = 0; $tablesNumber = 0; // Check status code if ($result['status_code'] == 200) { // If parent API call returns 200, call a MySQL API $tablesNumber = count($result['content']['resource']); // The next line is to limit number of tables to first 5 to see the successfull run of the script //$result['content']['resource'] = array_slice($result['content']['resource'], 0, 5, true); foreach ($result['content']['resource'] as $table) { // Get all fields URL $url = "<API_Namespace>/_schema/" . $table['name'] . "?refresh=true"; $result = $get($url); if ($result['status_code'] == 200) { $tableCount++; foreach ($result['content']['field'] as $field) { if (strpos($field['db_type'], 'date') !== false || strpos($field['db_type'], 'Date') !== false || strpos($field['db_type'], 'DATE') !== false) { // Patch field URL $fieldCount++; $url = "<API_Namespace>/_schema/" . $table['name'] . "/_field"; // Skip fields that already have the function if ($field['db_function'][0]['function'] === "TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM')") continue; // Remove broken function $field['db_function'] = null; $payload = ['resource' => [$field]]; $result = $patch($url, $payload); // Add correct function $field['db_function'] = [['function' => "TO_DATE({value}, 'DD-MON-YY HH.MI.SS AM')", "use" => ["INSERT", "UPDATE"]]]; $payload = ['resource' => [$field]]; $result = $patch($url, $payload); if ($result['status_code'] == 200) { echo("Function successfully added to " . $field['label'] . " field in " . $table['name'] . " table \n"); \Log::debug("Function successfully added to " . $field['label'] . " field in " . $table['name'] . " table"); } else { $event['response'] = [ 'status_code' => 500, 'content' => [ 'success' => false, 'message' => "Could not add function to " . $field['label'] . " in " . $table['name'] . " table;" ] ]; } } } \Log::debug("SCRIPT DEBUG: Total tables number " . $tablesNumber . " -> Tables " . $tableCount . " fieldCount " . $fieldCount); } else { $event['response'] = [ 'status_code' => 500, 'content' => [ 'success' => false, 'message' => "Could not get all fields." ] ]; } } } else { $event['response'] = [ 'status_code' => 500, 'content' => [ 'success' => false, 'message' => "Could not get list of tables." ] ]; } return "Script finished";
Email Alert API Usages
If there is a suspicious amount of API usage, send an email notification to an admin. This is a custom scripted service that uses a 'TransactionHistory' table to count the number of API calls in the last 5 minutes. An email alert will be sent to the specified email address if the number of API calls exceeds a specific threshold. For this example we are monitoring the usage of the API for the past 5 minutes and setting a threshold of 1,000 API calls. To modify these values to fit your project just change the 'timelapse' and 'threshold' values.
Note: This script can also be scheduled to run periodically using DreamFactory's built in Scheduling feature.
function twoDigits(d) { if (0 <= d && d < 10) return "0" + d.toString(); if (-10 < d && d < 0) return "-0" + (-1 * d).toString(); return d.toString(); } var toMysqlFormat = function (x) { return x.getUTCFullYear() + "-" + twoDigits(1 + x.getUTCMonth()) + "-" + twoDigits(x.getUTCDate()) + " " + twoDigits(x.getUTCHours()) + ":" + twoDigits(x.getUTCMinutes()) + ":" + twoDigits(x.getUTCSeconds()); }; var timelapse = 5; // How many minutes to count var threshold = 1000; // How many API calls are allowed var query = "timestamp%3c" + toMysqlFormat(new Date(Date.now() - timelapse * 60000)); result = platform.api.get("db/_table/TransactionHistory?filter=(" + query + ")"); if (result.content.resource.length >= threshold) { var email = { "to": [ { "name": "John Doe", "email": "[email protected]" } ], "subject": "Usage alert", "body_text": "API usage has exceeded the alert threshold.", "from_name": "John Doe", "from_email": "[email protected]", "reply_to_name": "John Doe", "reply_to_email": "[email protected]" }; platform.api.post("email", email); }
Create Custom Error Messages
You may wish to customize error messages if you are validating input parameters or simply want to guide the end user with a fleshed out response. To achieve this you can add a pre-process event handler with the code below.
// PHP throw new \Exception('Custom Message'); // Python raise ValueError('Custom Message') // NodeJS throw "Custom Message";
NodeJS script to POST records to a Database
This script simply creates a new record in the database table. Each time a GET call is made on the scripted service API endpoint, this script creates a new record in the table, 'departments'. You can create/write this data into any popular database whether it be MySQL, Microsoft SQL Server, or Oracle, or any other DreamFactory Service.
To create a scripted service, navigate to the 'Services' tab in your instance, hit the create button on the left, and select service type 'Scripts', and then select the desired scripting language. Don't forget that performing CRUD operations is just scratching the surface, you can also track things such as timestamps, status codes, and more!
// In the scripts tab, add this script to user -> user.session -> post -> user.session.post.post_process. // Make sure you've correctly configured Logstash as a service in the Services tab of the DreamFactory Admin Console. // To enable Node.js scripting, set the path to node in your DreamFactory .env file. // This setting is commented out by default. // // DF_NODEJS_PATH=/usr/local/bin/node // // Use npm to install any dependencies. This script requires 'lodash.' // Your scripts can call console.log to dump info to the log file in storage/logs. var url, result, options; options = { 'headers': { 'X-DreamFactory-Api-Key': platform.session.api_key, 'X-DreamFactory-Session-Token': platform.session.session_token, 'Content-Type': 'application/json' } }; var payload = { "resource" : [{'dept_no': 'd325', 'dept_name': 'Accounting'}] }; function createInternal() { // create a record using internal URL url = 'mysql/_table/departments'; platform.api.post(url, payload, null, function(body, response) { result = JSON.parse(body); }); } createInternal();
Validating API Payloads in Python
When using the /_table/{table_name} endpoint you can insert a new record into a database backed by a DreamFactory generated database API. By default the underlying database constraints will serve as a validation backstop, meaning if a required (not nullable) field is not provided, the database will throw an error and DreamFactory will in return return a 500 status code to the client. Alternatively, you might however wish to customize both the validation logic and error response. This can be achieved by using DreamFactory's scripting engine. To do so you can add custom logic to the POST endpoint's pre-process event handler, meaning the custom code will execute before the destination data source is contacted.
payload = event.request.payload if(payload): if 'first_name' not in payload: raise ValueError('First name field missing') if payload.first_name == '': raise ValueError("First name field required")
Python API Proxy
This can be attached as a post-process script to intercept the parameters being passed to the API call. The script then takes those parameters and passes it to the second API call. The data from the second API call is then returned.
# Retrieve the parameters that are being filtered from original API call params = event.request.parameters.filter; # Call second API and attach the same parameters to the API call result = platform.api.get('<service_name>/_table/customer?filter'+params); # Set the data to be returned data = result.read(); event.response.content = data;
What is CORS?
APIs are bridges useful for accessing otherwise siloed data from a variety of clients such as websites and mobile apps. However one must be careful to not give up security in the quest for convenience. Imagine an Internet where any third-party domain could initiate cross-domain requests which interact with your APIs and embed images and other assets hosted on your web server into their own site. All sorts of issues would quickly arise, including data theft and the ability to masquerade as a trusted resource of your organization. In fact, the prospects of this ability are so dangerous that all modern browsers prevent script-driven cross-domain requests by implementing what's known as the "same-origin policy".
At the same time, it would be incredibly convenient if this safeguard couldn't be overridden! Imagine the trouble associated with managing your APIs and all client websites on a single domain. Indeed there is a way to loosen this restriction using something known as CORS, or cross-origin resource sharing. When enabled, CORS will inform the web server to add supplemental HTTP headers which tell the browser how and in what ways the same-origin policy can be loosened. The most important such header is ‘’’Access-Control-Allow-Origin’’’, which tells the browser what other domains are allowed to request resources from the destination server.
The DreamFactory platform includes point-and-click support for securing your APIs with CORS. Check out chapter 8 of our Getting Started with DreamFactory guide to learn more.
Call a Stored Procedure From REST API
Although the DreamFactory platform can do many things, its ability to generate full-featured REST APIs for twenty databases (MySQL, SQL Server, and MongoDB, among others) is undoubtedly the most popular feature. Once generated, developers can review the API using companion Swagger documentation made available through the platform's administrative interface. Endpoints are available for carrying out the typical CRUD (create, retrieve, update, and delete) operations, inspecting the schema, and executing stored procedures. The URI signature looks like this:
’’’/_proc/{procedure_name}’’’
Therefore if the stored procedure name is ’’’getCustomers’’’, you can call the following URI:
’’’/_proc/getCustomers’’’
Many stored procedures accept input parameters. DreamFactory supports this functionality as well. You can pass the parameter along as a URL parameter, like this:
‘’’/_proc/getCustomerByLastName?LastName=Moreno’’’