Overview - Darth Vader Resorts

---

Darth Vader Resorts is a rental marketplace covering vacation rentals in 190 planets. It is a leading lodging company with more than 6,000 properties and reporting revenues of more than 17 trillion Imperial credits in fiscal year 2017.

All the data from the resorts across the planets is funnelled into the data center on Earth and has to be standardized and validated before getting consolidated into a DWH environment for reporting. Due to non-standards across the planets, major delays are seen in processing times and financial reporting. The financial calculations results in errors and improper projections. Lots of manual work is required to verify the financial information, resulting in lack of trust in the financial reports.

Darth Vader Resorts wants to implement a new solution that will reduce the cost and eliminate the delays of processing while streamlining the whole data consolidation process.

Successful implementation will allow Darth Vader Resorts to increase its financial reporting efficiency and improve their marketing campaigns, resulting in higher occupancy rates and increased financial gains.

Key Requirements

---

To achieve a successful rollout of the new system with all consolidated data the following requirements have been identified:

• Each planet sends a file every 12 hour
• Each file size varies between 2-4GB
• Each file should be processed under 15 mins; parallel processing and scalability is a key
• The platform should always be available 24x7x365
• Data growth is expected to be 5% YoY
• Developers are located on Earth and the testers are on Tatooine
• The three systems involved are:

Enterprise DWH

Reporting

Finance

Success Criteria

---

Overview - Hutt Building Materials

---

Hutt Building Materials is a builders merchant that supply materials to trades, nerf herders and the public across all arms of the galaxy. Their biggest customer is the Imperial Empire (now the First Order plc). Times are good for Hutt Building Materials as the Empire has been undertaking a series of large scale building projects, in particular:

• Two death stars (have you any idea how much plumbing a death star requires?)
• One Star Killer base – the electrical systems that consume a sun to power a giant death ray are a Hutt speciality

With more than 1 million different products in the catalogue, the business is under pressure to supply the right product attributes for each product. Dispatching the wrong products to the wrong base, planet or death star is resulting in delays to those projects. Hutt Building Materials is embarking on a project to improve the data quality of the product information in their catalogue. Accurate product information is key to cut down on wastage, loss of profit due returns, and risk of losing the contracts to the above projects.

Problem Description

---

Hutt Building Materials wish to receive product information from their suppliers in a number of different ways:

• Flat files received via an FTP
• One product at a time via a web service
• A batch of products via a web service

It is also possible for Products to be created / updated / deleted directly in the catalogue system by Hutt Building Materials employees. Product data needs to be validated as follows:

• Check that all required product attributes are present and valid and comply with all DQ rules
• Some data fields will need to be derived from values sent from the supplier. For example: if a product is listed by the supplier as being the marketing colour ‘Darth’s Helmet Noir’, this marketing value would be held as an attribute of the product, but there would also be a colour held for filter / search / categorisation purposes: in this instance ‘Darth’s Helmet Noir’ would become ‘Black’. These lookup tables need to be persisted somewhere and able to be managed by selected Data Owners via a user interface.
• Check that the product does not already exist, even where descriptions do not exactly match

Key Requirements

---

• Current system has 1 million products and each product has more than 20 different attributes.
• 100 new products expected on average per day.
• Onboarding a new supplier will require uploading their entire catalogue of up to 10,000 products.
• There are 20 product owners who arbitrate and overlook the description of the products in the catalogue.
• The 20 product owners can arbitrate a maximum of 100 data tasks each per week.
• Product information will need to be propagated and synchronised to other systems in real-time.
• An audit trail should be maintained on all Product data changes (CRUD)
• The solution needs to meet the following design requirements:

Product data is centrally managed

Provide real-time interfaces for other systems to query the catalogue

Product data is updated in real time

Do not use batch file processing (as this is the current situation which is not working)

• Hutt Building Materials are starting a new Talend Development Team initially consisting of 3 people. Team members have a limited amount of ETL experience using a different tool. They are also engaging a small boutique Talend certified partner (3 developers) and wish to have a Talend TAM. An additional 3 people will be used for Talend operations.

Success Criteria

---

Overview - Kelen Biolabs

---

Kelen Biolabs, a zoological research company founded by Fayet Kelen on the desert planet of Pasher, has a patented model on reverse engineering biological products to make medications. By far the largest company on Pasher, it struggles in a highly competitive market using archaic legacy data systems and high cost software licenses.

Kelen Biolabs currently has a number of contracts with the Galactic Empire which must be maintained to ensure the company’s future success.

The existing legacy data systems have become despicably obsolete and their required software licenses are simply too costly. Currently these systems contain well over 10 Yottabytes (1 Yottabyte =1000 Zettabytes =1 trillion Terabytes) of data.

Kelen Biolabs has decided that they need to move off the legacy systems to a new modern platform that will save the company billions of Imperial Credits per year in operations and software license costs alone.

Key Requirements

---

Migrate Legacy Data Systems to Modern Technology

• The four legacy systems are:

  • Channel Sales & Marketing
  • Product Research, Manufacturing, & Live Testing
  • Customer Relationship Management
  • Finance

  Two new Enterprise Applications have already been selected to consolidate these business functions onto commodity hardware to be maintained on premise. These 2 systems are:

  • SA3-P-O
  • Galactic Financials

Kelen Biolabs must move data off of their four legacy data systems currently in use.  All reside on Mainframe computers using the R2DB2 database. To achieve a successful rollout of the new systems, the following features must be implemented:

• Support processing of data in definable ‘chunks’
• Support processing of data in parallel
• Support execution of jobs in parallel
• Support scalable job servers for execution plans with parallelization
• Support 10 Developers, 2 Data Stewards, 1 QA tester, & 2 DevOps
• Map Legacy Systems to New System data structures
• De-Dup common records found in Legacy System
• Transform datatypes where needed
• Identify and store secretly all records associated with ‘WieLat’, a secret food supplement

Success Criteria

---

Overview - MadisCorp

---

Universal Guardians Corporation has split into two entities, MadisCorp and Unidentified Corp.

Unidentified Corp wants to retain the IT Infrastructure and has given MadisCorp 12 months to migrate off their IT systems.  This is an opportunity for MadisCorp to start fresh with a next generation platform that will provide better capabilities at lower cost.

Universal Guardians was running a conventional on-premise Oracle Data Warehouse to store its corporate information.  As a result of this split, MadisCorp will be allowed to import all existing data from the data warehouse into their modern platform for the next 12 months. MadisCorp wants more business agility and flexibility through real time analytics and access to the data.  The business wants to build a new Enterprise Data Warehouse for reporting across ERP, Sales, Channel, etc.

Since there are more than 2000 reports on top of the existing Oracle Data Warehouse, MadisCorp wants to limit the risks of inaccurate reporting.  Hence, they want to leverage the same reporting tool, and just plug it onto the modern platform without having to rebuilt the reports.

This modern platform will run in parallel to the old one for a few months to make sure that both systems generate the same reports.  Hence, any data changes going into the Oracle Data Warehouse will need to be replicated in the modern platform.

Key Requirements

---

MadisCorp wants to build a new Enterprise Data Warehouse with the following requirements:

• Leverage a cloud based platform for the data warehouse. Currently the plan is to use Azure and On-Premise data centers
• Highly Available Platform with 99.99% uptime
• Scalable and flexible platform that will scale with the exponential increase in data being acquired from various sources
• Reduction of Infrastructure Total Cost of Ownership
• Batch and Real-time integration capability
• Process more than 1 Petabyte of data daily
• Have at least 300 users on the platform
• Provide self service capability for business users

Success Criteria

---

Overview - Pod Racing Tech

---

Pod Racing Tech is a pioneering provider of journey and location-based ‘smart driving’ services.  The company aim to revolutionize the Pod Racing industry by providing new B2B and B2C services leveraging a combination of Big Data, Telematics and Mobile technologies.

Some of the new smart pod racing car services are:

• Emergency Call 
• Usage Based Racing Models
• Sniper Avoidance Location Services
• Light Saber Delivery Services

There are 1.5 million pod racers all over the galaxy with a need for real-time feedback on the above services.  The sensor on these pod racers sends the data to a decentralized network of receivers located on each known planet across the galaxy, and satellite relays.  The company wants to push targeted notifications and alerts about weather, conflict areas to avoid, etc. in real time to any pod racers based on their location. 

Key Requirements

---

• Real time integration with location services
• Cloud based solution
• A data repository to hold all of the transactions for 5 days
• Aggregated metrics per month and year on the transactions
• Support 15 million transactions per day across the galaxy – 10 transactions with 1.5 million pod racers
• 60 GPS Sensor updates per hour/per car
• Sensor updates are streamed into the platform and the current sensor data is compared with the previous one to determine location and speed of the Pod Racer
• Pod Racer Tech can leverage data from other real time feeds like Jabba the Hut Emporium,  Tatoine Data Services and other third party vendors
• There will be 100 developers on this project broken into 5 teams of 20
• Highly available system
• Ability to determine the speed and direction of travel of the pod racer based on their GPS Sensor Updates.
• Usage of the platform is based on a subscription. Pod Racers Tech charge on a monthly basis.

Success Criteria

---

Overview - VMF

---

VMF is the fastest growing Mobile Virtual Network Operator (MVNO) in the market today with less than 12 months track record. They have outstanding reviews with 90% of their 700,000 customers  reporting that they are happy, or very happy, with the customer service the company provided. Launched in the United Kingdom, VMF also has offices in Australia, France, and India.

The MVNO business is very specific and there is no dedicated turnkey solution. Since it began operations, VMF has adopted Open Source technologies for both practical and economic reasons. Today, most of the information system at VMF relies on a LAMP (Linux, Apache, MySQL, PHP/Perl) stack. Their current architecture landscape is made up of web applications, CRM, back-office, and MySQL databases.

The outstanding customer reviews and quality of service provided by VMF is helping the subscriber base to grow at a rate of 20% per month. Their customer base has reached one million subscribers and their calls generate close to 5 million invoice tickets per day.

The senior management at VMF has decided to invest in implementing a Data Warehouse that will enable them to report on all data coming from the various regions, i.e. UK, Australia, France and India.

Key Requirements

---

• Build a modern platform for the Data Warehouse
• The source systems are:
• Finance
• CRM
• Contracts
• Calls, Messages and other Services Logs
• Social Media
• Flat files from various offices
• The Data Warehouse should handle unstructured data
• Data should be refreshed within 24 hours in the system for the daily reporting needs
• Highly available platform running 24/7
• All calls, messages, and other services logs should be stored for at least 18 months for compliance reasons
• The volume for the logs data is increasing exponentially as the business is a tracking data usage of all their subscribers
• The data warehouse will drive sales, finance and customer reports
• VMF will be using Tableau as their reporting tool

Success Criteria

---