# Installation and Usage for AWS Marketplace

## Delivery model

Build from an AWS-provided EMR Serverless Spark image, preserving its entrypoint and `hadoop` user. The Dockerfile copies DataAI JARs to `/opt/dataai/lib` and `/usr/lib/spark/jars`. Customers select the resulting Marketplace image when creating EMR Serverless Spark compute. No DataAI service runs.

## Production-candidate artifacts

Install these matching-version modules together:

- `dataai-spark-api-1.0.0.jar`
- `dataai-spark-quality-1.0.0.jar`
- `dataai-spark-core-1.0.0.jar`
- `dataai-spark-functions-1.0.0.jar`

The CLI JAR is optional. Spark and Hadoop are provided by the customer runtime and are not bundled by DataAI.

## Generic Spark usage

Add all required JARs to the Spark driver and executor classpaths using the target platform's supported library or `--jars` mechanism. Then call `DataAiPipeline` and the functions in `com.dataai.etl.spark.functions`. The complete inventory is in `FUNCTION_CATALOG.md`.

Customer code supplies `Dataset<Row>` inputs and decides whether returned DataFrames are written. DataAI performs no automatic persistence.

## Production

Deploy immutable version `1.0.0` only after Yanbor release authorization, commercial entitlement, and target-platform validation. Obtain it from the approved marketplace artifact, authenticated Maven repository, or customer artifact repository. Verify checksums and pin the exact version.

## Troubleshooting

- `UnsupportedClassVersionError`: select a Java 17-compatible runtime.
- Class not found: install API, quality, core, and functions at the same version.
- Scala linkage error: use a runtime compatible with Scala 2.12.
- Spark linkage error: validate against the documented Spark 3.5.x baseline.
- Permission failure: use customer-approved identity and storage permissions; DataAI does not bypass platform controls.
