Available on AWS Marketplace · Version 1.0.0
Buyer integration guide
The application retrieves passages and applies tenant, permission and current
version filters before invoking the reranker. Retain the original source text
and resolve returned IDs against the exact submitted pool.
Send one UTF-8 object with exactly these keys:
{
"query": "How do I update my billing address?",
"documents": [
{"id": "billing_steps", "text": "Open Settings, choose Billing, edit the address, and save your changes."},
{"id": "delivery", "text": "Delivery tracking appears in your order history."}
],
"top_k": 2
}
Use application/json, or application/jsonlines with one object per invocation.
For line-split batch transform, use SingleRecord, MaxConcurrentTransforms=1,
SplitType=Line, AssembleWith=Line and Accept=application/jsonlines. Multiple
JSON objects in a single inference request are rejected.
Limits: 1..16 documents, unique opaque IDs matching [A-Za-z0-9_-]{1,64}, a
nonblank query and document texts of at most 16,000 Unicode code points each,
32,000 query-plus-document code points in total, a 96 KiB HTTP body and 512 actual
tokens for each complete query/passage pair, including special tokens. Both
query and passage consume that pair budget. top_k is an integer from 1 through
the supplied count. There is no silent token truncation. Chunk long documents
upstream and evaluate how the chunking changes your retrieval task.
Output contains the model/version, original upstream revision, ordered results
with id, zero-based index and score, and pair_tokens in input order.
Equal scores break ties by ID. Logits are uncalibrated; do not interpret them as
probabilities, compare them as a universal confidence scale or invent a shared
answerability threshold. review_required=true, truncated=false and
answerability_measured=false describe the contract.
Errors: 400 malformed/schema-invalid requests, 413 body size, 415 media type or
content encoding, 422 pair token limit, 429 busy, 500 inference failure. SageMaker
may wrap container errors in a 424 ModelError; inspect OriginalStatusCode. Retry
transient capacity/transport failures with bounded exponential backoff. Correct
invalid input before retrying. Do not log sensitive request bodies in your client.
Deployment requirements
The qualified real-time instance is one ml.g5.xlarge. Real-time deployment
must explicitly select InferenceAmiVersion=al2023-ami-sagemaker-inference-gpu-4-1
on its production variant. Its driver 580 supports the image's CUDA 13 stack.
The default g5 host driver 470 is incompatible. The qualified batch instance is one ml.g6e.xlarge with
TransformAmiVersion=al2-ami-sagemaker-batch-gpu-535. Pass
Environment={"SAGEMAKER_CUDA_COMPAT_DISABLED":"true"} to the batch job to
disable legacy compatibility mounting. The image contains the pinned userspace
compatibility libraries. Both selected instance paths passed actual same-image native portability tests;
model-package batch certification completed. Other instances are unqualified.
Subscribe through the Marketplace listing, then use Continue to configuration
to copy the product/model-package ARN for us-east-1. Set
SAGEMAKER_MARKETPLACE_MODEL_PACKAGE_ARN in your own notebook environment.
The notebook checks the completed package, its Marketplace marker and the
published image digest before creating resources.
Qualification measurements are for us-east-1; other catalog regions have not
been qualified by Software Sushi. Use a SageMaker execution role in your own AWS account, enable
network isolation, and avoid data capture unless your own data policy permits
it. Model weights and runtime dependencies are bundled; inference does not
download models or call seller services. No buyer source text appears in the
response or server logs. Your application and AWS logging configuration remain
your responsibility.
An endpoint continues to incur AWS infrastructure and applicable software
charges while provisioned, even when idle. Delete endpoints after testing and
stop unwanted batch jobs. Deleting a local notebook does not delete AWS compute.
Support
Email info@softwaresushi.com with model version, AWS region, instance type,
request ID and a redacted error code. Do not include private source documents,
customer data or credentials. No response-time SLA is advertised.
The notebook includes disabled-by-default examples for endpoint creation, inference,
a line-split batch job, stopping an unfinished batch job, and exact-name cleanup.
Download the fictional JSON Lines sample and upload it to your own private encrypted
S3 input location before enabling batch creation. A local waiter timeout leaves
AWS resources running; inspect and clean up explicitly after a timeout.