Enterprise Steam Release Notes¶
Change Log¶
Version 1.7.0 (May 25, 2020)¶
Major: Added support for running Enterprise Steam inside Kubernetes cluster
Major: Added support for deploying and managing Driverless AI instances on Kubernetes
Major: Added support for ML-Autodoc for automated H2O model documentation
Web Client: Redesigned the Web client for managing H2O/Sparkling Water clusters and Driverless AI instances
Web Client: Added automatic H2O and Sparkling Water cluster sizing when launching clusters
Web Client: Added support for displaying and downloading YARN logs of H2O and Sparkling Water clusters
Web Client: Added support for displaying and downloading H2O driver logs
Web Client: Added support for displaying and downloading Sparkling Water spark-submit logs
Web Client: Added support for displaying lifecycle events of H2O/Sparkling Water clusters and Driverless AI instances
Web Client: Added support for launching copies of H2O and Sparkling Water clusters
Web Client: Improved validation when launching H2O/Sparkling Water clusters and Driverless AI instances
Python Client: New API client
R Client: New API client
H2O: Added configuration to add YARN job name prefix
H2O: Added support for importing engine from the server
Sparkling: Added support for importing engine from the server
Sparkling: Added configuration to add extra JARs to the classpath
Sparkling: Added configurable maximum idle and uptime for clusters
Documentation: New documentation for Python client
Documentation: New documentation for R client
Documentation: New documentation for using ML-Autodoc with Enterprise Steam
Installation: Simplified installation and upgrade
Installation: Moved initial administrator setup from CLI to the Web Client
Installation: Added separate directories for static assets and data to support high availability deployments
Dependencies: Upgraded dependencies
Dependencies: Replaced HAProxy with Traefik
Various other fixes and improvements
Version 1.6.7 (Apr 28, 2020)¶
Fixed issue with environments containing multiple Spark installations
Version 1.6.6 (Apr 20, 2020)¶
Added support for Hive in H2O and Sparkling Water
Added support for Sparkling Water engines 3.28.1.1 and higher
Added option to allow insecure XGBoost
Fixed parsing of Spark properties containing ‘=’
Fixed possible corruption of HAProxy configuration
Version 1.6.5 (Mar 2, 2020)¶
Fixed custom Hadoop temporary directory to be part of HADOOP_CLIENT_OPTS
Version 1.6.4 (Feb 12, 2020)¶
Fixed error in Hadoop Helper for Java 7 Hadoop distributions
Fixed error when creating Python environment with the same name as deleted environment
Improved validation and documentation for LDAPs with internal certificate authority
Improved documentation for Driverless AI config.toml override
Improved documentation for Driverless AI installation prerequisites
Version 1.6.3 (Jan 31, 2020)¶
Supports RSparkling 3.28
Use Hadoop Tmp Dir for Hadoop Helper commands to prevent slowdowns
Fixed an issue with old DAI servers preventing instances from loading
Fixed server panic with invalid LDAP over TLS configuration
Fixed initial LDAP config requiring server restart
Version 1.6.2 (Jan 20, 2020)¶
Supports Sparkling Water 3.28
Fixed LDAP over TLS
Fixed SAML token login in R client
Fixed H2O profile not saving YARN queues
Version 1.6.1 (Jan 14, 2020)¶
Fixed mutual TLS setup
Fixed Driverless AI version parser
Add datatable_strategy WRITE to default DAI launch config
Add Driverless AI instance password to the instance details
Docs: Updated copyright year to include 2020
Version 1.6.0 (Dec 18, 2019)¶
Complete UI overhaul of configuration page for administrators.
New administrator settings for individual products and deployment backends and other configuration.
Improved UI/UX experience.
Moved most of the settings from steam.yaml into the administrator configuration in Steam UI.
Configuration file steam.yaml is no longer required.
Enabling Hadoop backend now thoroughly validates all settings to ensure they are correct (Proxyuser, Kerberos, …).
Added ability to export and import configuration.
Several settings no longer require Enterprise Steam restart.
Added indicator when Enterprise Steam needs to restart to apply changes.
Added option to use existing Livy server for Sparkling Water as long as it is on the same host as Enterprise Steam.
HAProxy comes bundled and does not need to be installed separately.
Steam Hadoop Helper (Steam Manager) comes bundled and does not need to be installed separately.
Additional HADOOP_CLASSPATH options can be added when launching H2O clusters.
Added ability to download complete Enterprise Steam logs from the UI.
Added SAML configuration validation.
Added cacert option when connecting to Steam or H2O clusters via Python client.
Added option to ensure internal secure connections with H2O and Sparkling Water clusters.
Added warning when uploading non-Hadoop version of H2O engine.
Improved database migration.
Improved validation of H2O Launch Parameters.
Fixed bug where Sparkling Water could not launch when users reused cluster names.
Fixed bug where Sparkling Water could not launch when the cluster name contained spaces.
Fixed UI issue when uploading Python environments with the same name.
Fixed Python environment selection when no Python environments assigned.
Fixed auto-assigment of profiles to LDAP groups
Version 1.5.1 (Nov 21, 2019)¶
Fixed a database migration bug affecting new installations.
Version 1.5.0 (Oct 10, 2019)¶
Added ability to launch Driverless AI instances.
Added ability to use Conda environments with Sparkling Water
Version 1.4.11 (Oct 1, 2019)¶
Fix token authentication in Python client when SAML is used
Fix R and Python client when internal and external ports do not match
Support recent versions of Sparkling Water
Version 1.4.10 (Jul 19, 2019)¶
Added an ability to download H2O logs from Steam UI
Added a config option for custom KRB5_CONFIG (STEAM_KRB5_CONFIG)
Added support for templating in YARN queue names
Added a check to ensure the Spark version and Sparkling Water version match
Added a check to warn admin if Steam port is occupied
Improved fetching of H2O YARN job status to now use impersonation
Fixed h2osteam Conda package
Fixed issue when Sparkling Water cluster failed to start when SPARK_YARN_MODE property was set
Version 1.4.9 (May 21, 2019)¶
Added Conda packages for Python client
Added ability to delete multiple stopped clusters at the same time
Added Python PySpark path configuration to profile
Added YARN queue configuration to user setting
Removed the need for secondary external port
Configuration fields are now protected from trailing and leading whitespace characters
Updated pysparkling dependencies for Python2
Fixed an issue with YARN usernames being lowercased when they were not configured to do so
Version 1.4.8 (Apr 15, 2019)¶
Add support for RSparkling through Jupyter notebook
Add option to disable Jupyterhub
Expose internal address and node ID for H2O cluster in the Steam Web UI
Add option to specify path to custom Pyspark Python for Sparkling Water
Fix pysparkling dependencies for Python3
Updated engine upload backend
Version 1.4.7 (Apr 3, 2019)¶
Fix Sparkling Water proxy issue with upercase usernames
Improve uploading h2o-3 engines
Set SPARK_YARN_MODE correctly based on the hadoop distribution
Version 1.4.6 (Apr 1, 2019)¶
Added ability to choose H2O-3 Leader Node when starting a cluster
Added ability to control the number of clusters a user can spin per cluster profile
Added option to select default Sparkling Water backend
Added automatic redirection back to login with an expired session cookie
Added an ability to auto-assign Steam profiles according to SAML profiles
Docs: Add “Before you begin installation” section
Docs: Documented steam.yaml configuration options
Docs: Updated documentation
Fix an issue when Steam was hitting API endpoints of dead clusters
Fix and issue when hadoop-unjar files were not deleted from temp directory
Fix issue with upercase usernames and Sparkling Water on Hadoop
Version 1.4.5 (Mar 22, 2019)¶
Added Configurable Steam Web UI timeout (STEAM_WEB_UI_TIMEOUT_MIN)
Version 1.4.4 (Feb 20, 2019)¶
Make log file permissions configurable (STEAM_LOG_PERMISSIONS)
H2O: Communicate with cluster using leader node only
SW: Added support for Hive tables
SW: Disable Spark dynamic allocation for internal backend
SW: Bundle and distribute all pysparkling dependencies
LDAP group configuration is no longer mandatory
Bug fixes for Jupyterhub
Bug fixes for Sparkling Water params
Bug fixes for CDH5
Version 1.4.3 (Jan 29, 2019)¶
Added ability to provide secure HTTP headers (SERVER_STRICT_TRANSPORT, SERVER_X_XSS_PROTECTION, SERVER_CONTENT_SECURITY_POLICY)
Improved logging output, time in UTC
Accept encrypted TLS private keys
Log files permissions elevated for the group
Bug fixes for HDP Spark
Bug fixes for Sparkling Water integration
Bug fixes for Steam Python API
Bug fixes for LDAP connector
Bug fixes for Jupyterhub
Bug fixes for Steam Web UI
Version 1.4.2 (Jan 1, 2019)¶
Add ability to upload H2O engine jar from Python/R API
Remove cluster dialog waits until cluster has been shut down
Log successful logins
Log files permissions reduced
Steam nbextension shows “please wait” when changing clusters
Document setup and troubleshooting steps for Hadoop/Kerberos configuration
Bug fixes
Version 1.4.1 (Dec 3, 2018)¶
Enable Sparkling Water API from API
Hide disabled user with a checkbox
Display message if there are no clusters to show
Fix glibc dependency for steam binary
Fix documentation version
Add release notes to documentation
Version 1.4.0 (Nov 23, 2018)¶
Add Sparkling Water integration
Add SAML authentication
Add More detailed cluster profiles
New cluster overview
New launch cluster page
Add option to generate Personal access tokens