platinum eta features in softnas loud enterprise 3 · apache nifi: easily build custom dataflows...
TRANSCRIPT
P a g e | 1 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
Platinum Beta Features in SoftNAS Cloud Enterprise 3.7
Overview SoftNAS Cloud Enterprise (previously known as SoftNAS Cloud NAS) is an integrated global cloud data platform, combining a software-defined enterprise-class NAS virtual storage appliance, high-speed data transfer up to 20x, and data integration/replication allowing IT to manage and control data. Platinum Beta Features allow customers to have an early look at potential future SoftNAS Cloud Platinum features, and provide SoftNAS early feedback on the Beta features.
Platinum Beta Features
SoftNAS® UltraFast: High Speed Data Transfer SoftNAS® UltraFast™ provides high speed data transfer, accelerating storage network traffic up to 20x compared to TCP/IP. Unlike WAN acceleration products, SoftNAS UltraFast™ is an intelligent, self-tuning storage acceleration technology, based on UDP, that is designed to address latency, congestion, poor QoS and slow data replication speeds, without redesigning or changing applications, networks or storage. Bandwidth Scheduling enables throughput and bandwidth throttling to regulate network traffic
and prioritize bandwidth use.
P a g e | 2 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
Use Case – Extending Premise Storage to the Cloud With SoftNAS® UltraFast™ Beta, storing and retrieving files from remote cloud instances is
optimized. Data stored remotely can be retrieved quickly and easily, allowing your team to share
and access files with or from branch offices in remote locations without loss of performance.
Use Case – On-Premise to Cloud Replication SoftNAS® UltraFast™ Beta allows you to quickly migrate, replicate, or integrate data from any
source, including legacy systems, and transfer it to any destination, helping you maintain a
timely replication windows from your on premise data center to any public cloud storage.
Liberate your network resources during business hours by scheduling when to use UltraFast, or
when not to use UltraFast.
SoftNAS® UltraFast™ Documentation
SoftNAS® UltraFast™ Networking
SoftNAS® UltraFast™ Common Configuration Steps
SoftNAS® UltraFast™ Replication from Premise to Public Cloud
Deploy Source and Target SoftNAS® Cloud ™ with SoftNAS® UltraFast™ Instances
SoftNAS® UltraFast™ Video Playlist
SoftNAS® UltraFast™ Video
SoftNAS® Lift and Shift: Data Migration (powered by Apache NiFi)
The Lift and Shift feature of our Platinum Beta provides a wizard to configure migration of on-premises file data to the public cloud leveraging the power of Apache Nifi. Point you Lift and Shift configuration to existing NAS, SAN, file server storage volumes on premise, select your public cloud storage target, configure your target volumes/storage and begin moving your file data to the cloud. Lift and Shift will continuously replicate changes from the source to the target until you stop the Lift and Shift flow.
P a g e | 3 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
Use Case – Datacenter Closure You have decided to close a datacenter, but need to move your data to the cloud. You can use
Lift and Shift to automate the process of migrating data to the cloud and use continuous sync to
ensure your cloud replicated data stays in sync until you are ready to shut your datacenter
infrastructure down.
SoftNAS® Lift and Shift™ Documentation
Lift and Shift™ File Migration
Configuring Ports and Security Groups for Lift and Shift™
Configuring FlexFiles Settings
Lift and Shift™ of a Volume from Source to Target
View Lift and Shift™ Data Flow through NiFi
SoftNAS® Lift and Shift™ Video Playlist
Configuring Network Security Group on Amazon Web Services (AWS)
Configuring Network Security Group on Azure
Configuring Network Security Group on VMware
Configuring the FlexFiles Settings
Using an External FileServer
Lift & Shift Advanced Configuration
Lift & Shift Create Target Storage Wizard
P a g e | 4 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
NOTE: In order create more then 2 Lift and Shift Flows or 2 volumes within a Lift and Shift, the
/var/www/softnas/softnas.ini needs to be modified. The “maxFlows” controls the
number of Lift and Shift Flows allowed to be active. The “maxFlowVolumes” controls
the number of volumes that can be configured within a Lift and Shift Flow.
SoftNAS® FlexFiles Many to One NiFi Dataflow Template: Quick start to moving data from many sites to one central site Platinum Beta Features also includes a “Many to One” NiFi Dataflow Template. NiFi Dataflow Templates are generic NiFi dataflows that can be reconfigured and extended to meet your configuration and requirements. The “Many to One” template included as part of the Beta and is configured to aggregate data from 2 remote sites to a central location by default, but can be expanded to include more than 2 remote sites. You can import the “Many to One” template into the NiFi User Interface to reconfigure the generic template to match your configuration.
Use Case Example – Remote Office/Branch Office (ROBO)
Replicate data from remote offices to headquarters for compliance centralization or to provide a
centralized data point for business analytics.
Use Case Example – Data Aggregation With many-to-one replication, this Platinum Beta is ideal for consolidating data from remote
offices or factories to a single location and ensuring a "single source of truth" for centralized
analysis.
Use Case Example – Centralized Disaster Recovery Easily replicate data from numerous global locations to a centralized repository providing cloud based continuous data protection (CCDP). SoftNAS UltraFast scheduled bandwidth throttling, and throughput controls eliminate the backup window problem where data is being created faster than it can be backed up.
Many to One Documentation
P a g e | 5 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
FlexFiles: Many-To-One Configuration Template
Many to One Video Playlist
Configuring Network Security Group on Amazon Web Services (AWS)
Configuring Network Security Group on Azure
Configuring Network Security Groups on VMware
SoftNAS FlexFiles: FlexFile Settings
SoftNAS FlexFiles: Lift & Shift Steps 1 - 3
SoftNAS FlexFiles: Target Node Template Configuration
SoftNAS FlexFiles: Source Node 1 Template Configuration
SoftNAS FlexFiles: Source Node 2 Template Configuration
SoftNAS FlexFiles: Starting a Many to One Configuration
SoftNAS® FlexFiles One to Many NiFi Dataflow Template: Quick start to moving data from once central site to many remote sites FlexFiles Architect is a key feature of SoftNAS Cloud Platinum. It allows a customer to create their own customized solutions. To help customers get familiar/more comfortable with using FlexFiles
Architect and NiFi, we need to provide NiFi templates as a helpful starting point to guide the
customer to the added value of creating custom NiFi solutions to their business problems (we need to make using NiFi more frictionless). The 1 to Many Solution Templates is one of the early solution templates that will be delivered as part of the SoftNAS Cloud Platinum product, and as part of the
Cloud Platinum Betas.
The 1 to Many Template is designed to help a customer use FlexFiles architect to build out an
automated solution to move data created at a central location to many remote locations. The template is a tool to help the customer get moving in the right direction to build such a solution with FlexFiles Architect. The templates will be accompanied by 1 to Many centric documentation and supporting videos to aid the customer.
P a g e | 6 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
Use Case Example – Remote Data Publishing
Replicate data from a centralized location to remote locations around the world to ensure the
latest information is available.
One to Many Video Playlist
Configuring Network Security Group on Amazon Web Services (AWS)
Configuring Network Security Group on Azure
Configuring Network Security Groups on VMware
SoftNAS FlexFiles: FlexFile Settings
SoftNAS FlexFiles: Lift & Shift Steps 1 - 3
SoftNAS FlexFiles: Target Node 1 Template Configuration
SoftNAS FlexFiles: Target Node 2 Template Configuration
SoftNAS FlexFiles: Source Node Template Configuration
SoftNAS FlexFiles: Starting a One to Many Configuration
Apache NiFi: Easily build custom dataflows Apache NiFi is a development environment that provides a seamless experience between design, control, data translation, feedback and monitoring for development of custom data integration flows and is included in the Platinum Beta. The NiFi User Interface allows you to create your own custom data integration flows using both built in NiFi Processors and custom SoftNAS NiFi Processors.
P a g e | 7 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
NiFi Processor Library
As part of the Platinum Beta Features, SoftNAS has included some new SoftNAS built NiFi Processors that can be used with the extensive list of existing NiFi Processors to build custom flows.
Use Case Example – Big Data Integration
Ingest data from multiple sources and feed it into systems like Hadoop Distributed File System
(HDFS).
Use Case Example – Filter and Transform Large Datasets
You used the Many to One template to aggregate data from multiple remote sites, but now you
need to filter the data into multiple smaller datasets for several business units. Once you have
the smaller filtered datasets, some of the data needs to be transform from a XML format to
JSON so the business unit’s applications can consume the data.
Apache NiFi Documentation
Getting Started with Apache NiFi
Scope of this Beta
Platinum Beta features are being delivered to allow customers to try potential future SoftNAS
features and provide “real world” feedback on them before they are generally available. The
scope of the beta will include movement of both block and object data from on premise to a
cloud platform and between cloud platforms.
P a g e | 8 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
Source Node Type Target Node Type
Supported Target Node Storage
VMware AWS EBS, S3
VMware Azure Premium & Standard Disks
AWS AWS EBS, S3
AWS Azure Premium & Standard Disks
Azure AWS EBS, S3
Azure Azure Premium & Standard Disks
SoftNAS requests beta participants help SoftNAS by provide specific details regarding their
Platinum Beta experience. Your input will help SoftNAS enhance the Beta features. The type of
input we are looking for includes:
Use case example to help us better understand your application for using these features
Simple configuration details including on-premise as well as cloud environment such as
type, size, storage types used etc.
What you liked as well as disliked
If we can follow-up for additional discussions
You can submit your input by sending an email to [email protected]
What is Not Recommended for this Beta
Using SoftNAS® UltraFast™ with Lift and Shift or the FlexFiles Templates.
Using Lift and Shift or FlexFiles Templates with Snap HA, SnapReplicate, Dual Controller
High Availability (DCHA).
Beta FAQ:
Can I use Platinum Beta Features in a Production Environment – The set of beta
features are to be used in a test environment only. Platinum Beta Features should not
be used in any production environment.
How to get support – For support on these beta features, please send an email to
[email protected]. Please include a detailed description of the issue. These beta
features are not meant to be used in a production environment and as such, support
will be collecting the details provided and take the appropriate actions. However, Beta
features are not covered under our SLA with support provided during normal business
hours for Silver support, 9am to 5pm US Central Monday-Friday.
What size instance should I use –
1. VMWare:
Network: 1Gb/sec (120MB/sec)
RAM: 28GB to 64GB
P a g e | 9 of 9 Copyright ©2018 SoftNAS, Inc Monday, January 22, 2018
CPUs: 6 to 12
2. AWS: m4.2xlarge or larger
3. Azure: DS4_V2 or larger
Beta Timeframe – The estimated duration for this beta is September 20th, 2017 to
March 31st, 2018.
1. SoftNAS reserves the right to end or extend support of the Beta
How long does it take for FlexFiles services to start after I accept the Beta prompt – It
can take 15-20 minutes. All Beta services are dormant until you acknowledge using the
Beta services, so it can take several minutes to get all the Beta services started for the
first time.
Is there a specific sequence of steps needed to set up the FlexFiles features – Yes
1. Set up the Network Security Groups for your SoftNAS Instances
2. Run the FlexFiles Settings command in the Storage Administration Pane within
the StorageCenter UI.
After these steps, you can either start the Lift and Shift feature, configure a Many to
One relationship, or access the Apache Nifi interface (FlexFiles Architect command
in the Storage Administration Pane).