Release Notes

Version 2.12.0 (Released February 25, 2018)

• Added Bitbucket.org support for Git services
• Added beta support for MemSQL (requires subscription)
• Added response caching for script services
• Fixed wildcard handling in session permission checks
• Use potentially modified request in response format handling
• Updated API documentation and bug fixes for file storage services
• Fixed some Oracle data types not getting caught for type simplification
• Disable date and time types formatting when there is no translations configured
• Added count_only option for Salesforce service, Fixed session timeout

Version 2.11.1 (Released January 25, 2018)

• Fixed NodeJS (and Python) script execution for large script to use a configurable script size
• Fixed CORS issues
• Added support for accented characters in api route
• Added support for accented characters in storage folder/file name
• Added support for multipart/form-data file upload
• Added implements_access_list config option for Script and HTTP service for overriding swagger def for role service access
• Fixed zip errors in package exporter using empty folders in remote file systems
• Fixed absolute path based local file service for MS Windows
• Modified local file service to pull from config so that it picks up DF_MANAGED_STORAGE_PATH for local files
• Fixed NodeJS (and Python) script execution for large script, making script size limit check configurable
• Fixed role service access components when swagger def is supplied
• Role service access components corrected in use of 'as_access_list' option
• Initial support for multi-column constraints
• Changed error message to indicate why DynamoDB schema overwrite doesn't work

Version 2.11.0 (Released December 29, 2017)

• Upgraded Laravel Framework to 5.5, added package discovery to all dependencies.

Note: This and all further releases require a minimum PHP version of 7.0

• Added support for MongoDB GridFS as a file storage system
• Added support for RabbitMQ (AMPQ) pub/sub protocol (requires subscription)
• Added support for Facebook's GraphQL API for database services (beta)
• Added alternative means of authentication using external database connection
• Added ability to set max_records_returned per database service instead of system-wide
• Updated routing to allow for configurable api and storage route and make version optional (api vs api/v2)
• Fixed OAS3 handling of comma-delimited URL query parameters
• Fixed issues with content type handling
• Fixed empty header issue in cURL response in remote HTTP response handling
• Dropped for 'token' as a URL parameter, use 'session_token' or header
• Internal changes to move 'system' service to its own repo (df-system)

Version 2.10.0 (Released November 6, 2017)

• Upgraded from Swagger 2.0 to OpenAPI 3.0 specification for API Docs and generated service definitions

Note: This is a breaking change for HTTP (aka Remote Web Services) and Script services using API Doc service definitions (i.e. Swagger 2.0 specifications). Please update your JSON or YAML service definitions to the latest OpenAPI Specification 3.0.0 or above. If your definitions have the service name included in the path designations (/service_name/my_endpoint), please remove them (/my_endpoint) as this is no longer supported. There are a couple of online tools here and here to convert your old files to the new format. Or go here for local development tools to convert.

• Admin application now usable by non-admins, access to each tab is based on RBAC privileges
• Added attachment support for email services and email templates
• Added ldap_username to lookup availability
• Added support for NTLM authentication for SOAP services
• RBAC support for system/package endpoint for export or import
• More detailed error message for Access Forbidden (403)
• Tailored system/environment call for various authentication levels
• Corrected RBAC view of api/v2 service listing
• Limit resource listing displayed fields when discovery is not complete (database tables and procedures)
• Fixed scripting use of lookups
• Fixed Node.js (and Python) scripting error due to big data set in scripts
• Fixed several integer casting issues with service configuration when using SQLite as the system database
• Moving RBAC exceptions and license and subscription requirement handling to the actual services domain
• Removed app_group system resource and supporting classes, no longer used
• Updated homestead configuration
• Added CloudFoundry manifest/setup files
• Updated unit test cases for some subsystems

Version 2.9.0 (Released September 19, 2017)

• Added support for Active Directory Single Sign On and SQL Server Windows Authentication
• Added support for Github and GitLab source control services, with file linking from server-side scripting
• Added support for file service file linking from server-side scripting
• Added database record caching for retrieved records, enable with a TTL in the service config, purge or refresh on demand
• Added support for server-side filters on some NoSQL databases (MongoDB, DynamoDB, Cassandra, others coming soon)
• Added support for HAS_ONE relationship, similar to HAS_MANY but where the FK also is the PK or has a unique constraint
• Enhanced usability of the base api/v2 output for available services
• Fixed server-side filters so that they adhere to the requestor type API or Script
• Allow setting DF_SCRIPTING_DEFAULT_PROTOCOL in .env to "http" (default) or "https" for internal API calls from Node.js and Python scripts
• Fixed handling of parameters when stored procedure is not on the default schema

Version 2.8.1 (Released August 18, 2017)

• Exchanging df-swagger-ui for new df-api-docs-ui API Docs application
• Update FileManager application to use new APIs
• Added clearer validation messages
• Service list retrieval and better service handling in ServiceManager
• Add caching to base service and ability to clear at the service level
• Add ability to log database queries, enable via .env
• Reworking API doc usage and generation
• Moving API docs permission check for role-level swagger control
• Only return debug trace when app.debug is true, previously used APP_ENV
• Rework schema interface for database services in order to better control caching
• Rework access check to always return JWT errors if a token is given
• Catch and log service exceptions during event list generation
• Fix swagger definitions to pass validation checks
• Make sure we have a status code on exception handling
• Fix lookup creation and validation against existing lookups
• Fix lookup hierarchy in session information

Version 2.8.0 (Released July 28, 2017)

• Support for FTP, SFTP, FTPS, and WebDav as file services
• Support for IBM Informix as a database service
• Support for MQTT project as a service (new IoT service group type)
• Support for Limit Exceeded event scripting for API Limits feature
• Added SAML and OpenID Connect SSO support
• Added service/resource info on logstash logging
• Cleanup to allow easier pulling of event map and API docs from services
• Rework Event structure to allow for non-API driven service events
• Cached cors config
• Rework lookup storage and modeling to ease session usage
• Do not include service in event list if no events generated
• Add potential resource handling and Swagger doc info gathering at the base service level
• Allow empty lookup values
• Made DELETE behavior consistent across local and remote file services
• Creating Belongs_To and Many_Many relationship records correctly in database services
• Better handling for date and time fields including microseconds and timezone adjustments
• Fix boolean type filters in Postgres
• Support PDO constants directory in database config options
• Fixed migrations and other issues for SQL Server when using the DBLIB driver
• Added support for new regions in AWS services
• Fixed S3 file listing when number of files is too large
• Added username field for LDAP integration to shadow user
• AD/LDAP bug fixes (ignore fetching object by DN from child domain, ldap login when uid field (username) is blank)
• Added DF_JWT_USER_CLAIM env option to include user attribute in JWT
• Fixed vagrant box provisioning script for Cassandra driver

Version 2.7.0 (Released June 06, 2017)

• New BASH installer.sh for customizing environment and packages included
• Support for OpenID Connect
• Support for CSV as a data source
• Added email service and template configuration to "system" service for admin invite and password reset
• Many Admin application improvements
• Cleanup after removing php-utils library dependency
• Cleanup existing php_unit test infrastructure
• Added license level in Config - System Info page
• Added server external IP address on system/environment
• Moved JWT require from application to df-core
• Extra user and admin info on package manifest
• Fixed API Docs to show token refresh endpoints
• Fixed GET over POST on system resources ignoring some parameters in payload
• Allowed email template to enter multiple addresses in to, bcc, cc fields
• Fixed package import with overwrite for currently logged in user
• Fixed user authentication after changing user email and password from profile page
• Added better error message for password change failure
• HTTP Services now look for headers from request to pass through when called from scripting environment
• Fixed CSV and XML user import feature
• Fix migration for MS SQL Server possible cascading issue
• Remove cipher option of RIJNDAEL-128 from environment
• Used user_key in JWT claim from improved security
• Fixed "count_only" parameter usage in database services
• Limit feature no allowed endpoints to be saved as null for evaluation as top level resource
• Fixed OAuth issue with PHP 5.6
• Split df:setup command into df:env and df:setup

Version 2.6.0 (Released April 21, 2017)

• Added support for username based authentication via environment configuration option, email is still default
• Added support for Firebird SQL Database
• Added support for specifying endpoints and verbs on Limits feature
• Added support for using Redis as cache for Limits feature
• Added support for "upsert" on database services where supported (using PUT verb)
• Added support for Admin user email invites
• Fixed date and time configuration option usage for database services
• Fixed inconsistent behavior regarding selected fields and related data in SQL database services
• Fixed JWT token refresh and handling forever token refresh properly
• Ignore received JWT explicitly when making login requests
• Fixed resource-level role-based access control for API docs
• Fixed PUT section on API docs for system/{resource} and system/{resource}/{id}
• Cassandra service updated to better handle native types like UUID and Timestamps
• For HTTP services, added support for detecting and replacing external links in response when resolved to other DreamFactory service URIs
• Fixed scripting response handling with single quotes, particularly in python
• Fixed scripting execution with exceptions in Node.js
• Fixed scripting handling of file upload

Version 2.5.0 (Released March 4, 2017)

• Upgraded and restructured to latest Laravel 5.4 framework

Note: As part of the upgrade, the default charset and collation have been changed to utf8mb4 to better support UTF-8. This will cause a problem for brand new installs on older MySQL-based databases, MySQL before 5.7.7 or MariaDB before 10.2.2. If you using those databases, uncomment or add the following lines in your .env file.

##DB_CHARSET=utf8
##DB_COLLATION=utf8_unicode_ci

• Added support for new Limits feature
• Added support for new iOS and GCM Push Notification feature
• Changed database error message, particularly with batch requests, to be consistent across services
• Fixed AD login issue with accented characters in user names
• Upgraded to latest AWS SDK, used latest internal filtering for DynamoDB
• Fixed schema issues with AWS Redshift
• Fixed several issues with Apache Cassandra from earlier beta release
• Added overwrite option for package import
• Fixed migrations with timestamp fields due to Laravel issue #11518 with some MySQL versions
• On incoming XML, handled any outer wrapper, not just 'dfapi', as there is no need to restrict
• Script tokening to authenticate internal script calls from Node.js and Python scripting
• Fixed a package export error
• Fixed creating role where description is longer than 255 characters
• When using 'ids' in URL or payload, always return a batch response, even for a single id

Version 2.4.2 (Released January 17, 2017)

• Added SAML 2.0 support as an authentication service
• Added support for Azure Active Directory
• Added support for Homestead ^4.0 with PHP7.1
• Added event-driven configurability to logging service
• Added database function support across all fields, not just virtual, so we can support binary and unknown data types
• Added event.setRequest() for Node.js scripting
• Clearer naming of log context options in logger service
• Prefer Microsoft pdo_sqlsrv driver over pdo_dblib for SQL Server on Linux-based OS
• Updated v8js support on Homestead for PHP7.1
• Updated Vagrant provision script
• Refactored out database, email, and script services to their own repos from core
• Fixed verb tunneling for proper role access control
• SMTP service no longer requires authentication or SSL/TLS
• Clear cached WSDL files from SOAP services upon system cache clear
• Better error handling and showing original content when content in response cannot be resolved to Accept type
• Package management improvements
• Indicating bad services (connection failures, etc) on package manager
• Schema management improvements
• Fixed use of special words like "table" in SQLite
• OAuth callback handler now checks for service name using state identifier when service name is not present on callback url
• Handling exceptions thrown in callback functions in NodeJS scripting
• Added send_invite parameter to API definition documentation
• For various model members, If value is null, don't return protected mask or encryption, just null
• For database services, check column count so we don't attempt a fetch on a non-existent rowset
• SOAP services now load WSDL files from storage/wsdl if only file name given

Version 2.4.1 (Released December 1, 2016)

• Dependency update for df-admin-app (Admin application) changes

Version 2.4.0 (Released November 18, 2016)

• Added Couchbase database service type (dreamfactory/df-couchbase)
• Added Microsoft Azure DocumentDB service type (dreamfactory/df-azure)
• Added Lookup modifiers and modifier configuration options, i.e. {urlencode(my_lookup)}, see wiki.
• Added a pre-configured local file service for the logs directory
• Added parameter and header options to scripting inline calls using platform
• Added support for schema merge in package import
• Reworked Virtual Foreign Keys, now called Virtual Relationships, a.k.a. DataMesh, to support all relationship types, including provisioning/retrieving from MongoDB services
• Added new API paths for database table field and related management (_schema/<table_name>/_field and _schema/<table_name>/_related). Old field access with _schema/<table_name>/<field_name> will be deprecated in later releases
• Added ability to define fields for MongoDB tables (primarily for Virtual Relationships but will be expanded for validations, etc. in coming releases)

• Fixed AWS SNS service API issues
• Update CORS to use the latest laravel-cors with additional options and new path matching
• Removed array wrapping of event.request.headers values
• Use null for empty service doc instead of default JSON object
• Protecting user, role, and app lookups against duplicate named entries
• Preventing timeout on package export manifest by only showing top level folders for file services
• Node.js and Python scripting improvements
• Don't format null to defined param type on stored procedure output parameters
• Clean up database extras upon dropping table or column
• Casting boolean values correctly for Sqlite, IBM DB2

Version 2.3.1 (Released October 4, 2016)

• Added configurable role per app for open registration, OAuth, and AD/LDAP services
• Added Log service supporting Logstash (commercial packages)
• Added AWS Redshift service support (beta)
• Added Python command path setting to distributed env file
• Added 'count_only' option to query parameters to return count of records filtered, but not the records

• Made user registration and password reset confirmation code length configurable
• Made user registration and password reset confirmation expiration configurable
• Made file services support chunking for downloading large files
• Changed core models to allow for encryption and protection control
• Changed passwords, secret keys, etc. in service configurations to be protected, i.e. can be set but not retrieved through API
• Cleaned up cached Service model usage
• Make server-side filter usage case-insensitive like the rest of record processing
• Improve related data queries by pushing full dataset down to per relationship handling

• Changed OAuth services to return token after OAuth authentication, improved consistency across all OAuth providers
• Added WSDL and Salesforce REST API version selection options for service config
• Added example WSDL file for Salesforce connections
• Added OAuth config options to Salesforce service access via Salesforce OAuth
• Session token management for Salesforce API now cached and handles both authentication modes

Version 2.3.0 (Released August 23, 2016)

• Added Cassandra database service type (dreamfactory/df-cassandra)
• Added Cache service type, currently supporting local configured cache as well as Redis and Memcached configurations (dreamfactory/df-cache).
• Added Queue-able option to system event scripts (<service>.<resource>.<verb>.queued) and script service types for queueing scripts to run after processing of API requests.
• Added Microsoft Live OAuth2 option to OAuth service type.
• Added 'doc' to every service listing (api/v2/system/service), replacing optional relationship parameter.
• Added 'is_base64' option for retrieving content of file along with properties for file services.

• Moved event script CRUD operations to 'api/v2/system/event_script', 'api/v2/system/event' now only lists events, or only scriptable events when 'scriptable' parameter set to true.
• Allows post-process scripting to always run, even when processing throws exception.
• Allows pre-process to circumvent processing request by returning response directly.

• Fixed public path trailing slash usage on file storage services.
• Fixed scripting bug where the system failed to check a script file path.
• Fixed showing wrong disk name for local file service container config options.
• Fixed stored procedure response wrapping issue.
• Fixed Node.js remote calls issue when URL has port in it.
• Cleaned up the update schema handling to avoid sending unnecessary changes to database.
• Workaround for v8js segfault issue in PHP 7.0.

Version 2.2.1 (Released July 11, 2016)

• SQL Database Stored Procedure and Function access and usage enhancements. HTTP GET using the "ids" parameter returns details about procedure and function requirements. Now call procedures and functions with just the required parameters. Also added the ability to pass procedure and function parameters as URL parameters, inline, or via a "params" array in the payload.
• Added Active Directory support for group hierarchy when mapping to DreamFactory roles.
• Added event modification configuration setting to pre- and post-process scripts, replacing usage of "content-changed" flag in the script itself.
• Added data import support for packages, initial support for database record importing.
• Added event matching from Swagger documentation paths to support event firing on exact and matching paths for HTTP Services.
• Added event support for individual SOAP methods defined in the given WSDL.

• Fixed Swagger model generation for SOAP service, now supports WSDL enumerations in types.
• Fixed file streaming using file service over CORS connections.
• Fixed role exporting.
• Fixed table export issue in packaging feature.
• Fixed encoding of resources forwarded on in HTTP Services.

• Caching fixes and speed improvements.
• Enhanced error message for some APIs, particularly in authentication and authorization areas.
• Admin application UI tooltip enhancements.
• Cleanup of API Docs.
• General cleanup of usage of utility functions.
• Testcases reworked for more dynamic services.

Version 2.2.0 (Released May 31, 2016)

The following services are now under a commercial license and have been removed from the default installation: AD/LDAP, SOAP, Salesforce, MS SQL Server, SAP SQL Anywhere, Oracle, and IBM DB2. Please contact [email protected] for further information.

• Redesigned services, script engines, and system resources management to be more flexible and dynamic. Now using ServiceProviders for all service type on-boarding.
• New service type migration command for pre-2.2 database upgrade (php artisan dreamfactory:service-type-migrate), run after migration and seeding.
• SQL DB driver types now available as their own service types, "sql_db" type retired.
• Script languages now available as their own service types, "script" type retired.
• Converts old services types to new format during import in packaging.

• API Doc now supports OpenAPI (fka Swagger) YAML format, as well as JSON.
• Service Definition system now adds service name to all defined paths and tags automatically
• Support for service definition (Swagger doc) on service import/export in packaging.

• Added platform.api support for Node.js and Python scripting
• Python scripting improvements, like allow empty script, correcting script output
• Node.js scripting improvement, like allow returning output from async callback functions

• Including predis/predis package by default for using Redis for caching
• Including df-azure using microsoftazure/storage by default, used sdk that required pear in prior releases
• Auto login enabled after creating first admin and other speed improvements
• Now using guzzle 6
• Added laravel/homestead support for php5.6 and php7 for dev installs

Version 2.1.2 (Released April 26, 2016)

• Redesigned Packaging feature, including new system/package API and artisan commands for import and export.
• Data Mesh feature now supports SQL to MongoDB virtual relationships.
• Added Redis and Memcached config options to dreamfactory:setup command
• Added Memcached config values in environment
• Updated dreamfactory:import-pkg command to use new package
• Updating initial setup to import any packages available
• Handling file stream output using StreamedResponse
• Added artisan dreamfactory:config-hhvm command to create hhvm config file
• Basic support for www-form-urlencoded payload in API requests
• Cleanup service providers
• Add rewrite rule to allow basic auth on AWS cloud and VM.

Version 2.1.1 (Released March 15, 2016)

• Upgraded Laravel framework to 5.2
• Enhanced setup command
• Now allowing login with JWT passed as URL parameter
• Added log level support, see DF_LOG_LEVEL in .env-dist for options
• Enhanced logging includes REQUEST and RESPONSE under log level INFO.
• New and improved File Manager application is now loaded from a composer-controlled repo
• API Docs (aka Swagger UI) updated for spec validation.
• Added extra server side and client side information on the config tab of admin app.
• Angular 2 and ReactJS sample apps added to listing in admin app.
• Updated Node.js scripting to support callbacks in scripts and log all console.log output to DreamFactory's log.
• Support non-DreamFactory (<dfapi>) XML wrapper on incoming data.
• Support for simplified DB filter operators "contains", "starts with" and "ends with"
• Lookups now supported in scripts. Lookup notations (i.e. {lookup_name}) get replaced before script is run.
• Lookups now supported in parameters for SQL DB stored procedures.
• Fixed a bug that prevented private lookup keys to be used in service credentials.
• Better support for spaces in database column names.
• API Docs for file services path root operations, adding back POST,PUT,PATCH,DELETE.
• Fixed support for SQLite as server database.
• Support for SQL Server legacy image type.
• Fixed a cache reset issue on user-app-role assignment.
• Updated AWS SDK version and support to latest 3.* release.

Version 2.1.0 (Released February 9, 2016)

Note: This release includes some breaking changes, not in the API itself, but with Live API Documentation (i.e. Swagger)!

• Upgraded usage of OpenAPI Specification (fka Swagger) to 2.0 from 1.2. Prior service configuration (i.e. for remote web services) or client dependencies (old swagger-based SDKs) on the previously used 1.2 specification are not compatible and must be changed to comply with the 2.0 specification.
• Swagger UI (supporting 2.0 spec) now included as an application and referenced in Admin application.
• API Doc (i.e. Swagger UI) view is now Role-based, meaning that the view is based on the allowed services for that Role.
• Event Scripting post-process scripts, and custom scripting services (V8Js, NodeJs, and PHP) now can utilize a "event.response" object that allows for custom status code and content type settings directly from script. Using the old-style "return" statements to return data still works and defaults to status code 200 as before.
• Event Scripting now supported for File Services and Remote Web Services.
• XML to JSON content conversion now handles namespaces (i.e. singular namespaces are used to pull data into JSON format).
• Fixed bugs with configuration of Email services.
• Fixed bugs for manipulating request content and parameters in pre-process scripts.
• Fixed handling for database "read-only" fields like SQL Server rowversion.
• SQL Server now supported as an option for the system database.

Version 2.0.3 (Released December 22, 2015)

• Updating composer to allow PHP >= 5.5, i.e. PHP 7.0 tested and working!
• Laravel Homestead configuration setup now part of install for quick testing.
• Added ability to update related records on virtual foreign relationships for same or different SQL DB
• Added alias editing via Admin console for Schema relationships
• Fixed PostgreSQL, SQL Anywhere and Oracle databases issues with table names
• Fixed API DB max records return limit usage
• Fixed several DB filtering issues
• Remove loading of lodash by default on V8js scripting, use require() instead if desired
• Fix stored procedure on MySQL issue when no data sets returned
• Fixing CORS config entries when using SQLite as system database
• Added 'computer' resource for AD/LDAP services
• Added LinkedIn OAuth 2.0 support

Version 2.0.2 (Released November 30, 2015)

• Added ability to create virtual foreign keys across two tables on same or different SQL DB
• Added aliases for virtual fields and relationships
• Added ability to create virtual fields for db-supported functions, e.g., concat(field1, field2)
• Added ability to create virtual fields for aggregation by field name
• Added support for assigning AD users to DreamFactory roles based on AD group assignments

For details please see the relevant change logs on GitHub.

dreamfactory/dreamfactory dreamfactory/df-core dreamfactory/df-sqldb