DataAI ETL AWS Glue / EMR - Customer-Ready Evaluation

Product: AWS Glue / EMR

Distribution: Customer-ready evaluation

Source: DataAI_ETL_AWS_Evaluation/docs/INSTALLATION_AND_USAGE.md inside DataAI_ETL_AWS_Evaluation.zip

Package SHA-256: 81235dc9475947c62e43eb96d9ad9ce6db298d9414f3655b9cbb470cf97e4245

Download customer-ready package | Download checksum | Read the evaluation license

Start here: Download one customer-ready ZIP. It already contains the DataAI libraries, product assets, documentation, samples, licenses, internal checksums, and local verification files for this integration. Spark, Hadoop, vendor products, and proprietary drivers remain customer-provided.

AWS Glue / EMR: Customer Evaluation Installation and Usage

This is the customer-ready DataAI ETL evaluation package from Yanbor LLC, provider of the DataAI product. DataAI runs inside the customer's pipeline; it is not a hosted service and makes no required DataAI network call.

Prerequisites

Java 17
Apache Spark 3.5.0 with Scala 2.12
Customer-controlled AWS account and Glue, EMR, or EMR Serverless evaluation runtime

Installation and first verification

Review LICENSE.md and run smoke-test\run-smoke-test.ps1.
Upload the DataAI JARs to a private S3 location and grant the job identity read-only artifact access.
Attach the JARs to the selected Glue, EMR, or EMR Serverless Spark job.
Run only against isolated evaluation sources and customer-selected outputs.
Verify IAM, networking, catalog, Spark/Scala compatibility, output rows, and matrix convergence on the exact AWS runtime.

Evaluation boundary

Use only fictional or approved non-production data.
Review LICENSE.md; the standard evaluation is less than 32 consecutive calendar days.
DataAI software is provided AS IS, with no obligations except those expressly accepted in a signed commercial agreement or order form.
Spark, Hadoop, cloud platforms, BI tools, orchestration products, and vendor drivers remain customer-provided and governed by their respective terms.
A successful local smoke test validates the delivered DataAI artifacts. It does not claim certification on the customer's external platform.

Package layout

install/: platform-specific installer or packages, when applicable.
lib/: DataAI JARs required by Spark-based integrations.
smoke-test/ or nuget-smoke-test/: executable local verification.
docs/: complete function catalog and product-specific guidance.
samples/: fictional evaluation inputs.
platform/: platform-specific examples, notebooks, configuration, POMs, or container-review assets.
PACKAGE_CONTENTS.sha256: SHA-256 for every other file in this ZIP.

Local DataAI Library Smoke Test

This test uses only fictional in-memory rows. It starts local Spark, executes normalization, profiling, required and numeric quality rules, clean/rejected routing, the complete function-catalog check, and iterative matrix balancing.

Requirements

Java 17.
Apache Spark 3.5.0 with Scala 2.12.
spark-submit available on PATH, or supplied with -SparkSubmit.

Spark and Hadoop are customer-provided runtimes and are intentionally not bundled in the DataAI evaluation ZIP.

Run

From the extracted package root:

.\smoke-test\run-smoke-test.ps1

If the customer's Windows policy blocks direct script invocation and the customer's security policy permits a process-only override, run:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File `
  .\smoke-test\run-smoke-test.ps1

This option applies only to that PowerShell process; it does not change the computer's execution policy.

For an explicit executable path:

.\smoke-test\run-smoke-test.ps1 `
  -SparkSubmit 'C:\spark\bin\spark-submit.cmd'

Success requires exit code 0 and a DATAAI_*_SMOKE_TEST_PASS marker. This local check validates the delivered DataAI libraries. It does not replace the documented validation on Databricks, AWS, Azure, Oracle, Google Cloud, Tableau, Alteryx, or another customer-selected external platform.

Verify the downloaded ZIP

$expected = (Get-Content .\DataAI_ETL_AWS_Evaluation.zip.sha256).Split()[0]
$actual = (Get-FileHash .\DataAI_ETL_AWS_Evaluation.zip -Algorithm SHA256).Hash.ToLowerInvariant()
if ($actual -ne $expected) { throw 'DataAI ETL package checksum mismatch.' }