Senior Data Scientist (Generative AI)

Senior or Principal Data Scientist (Generative AI)About the role We are looking for a hands-on data scientist to build and lead the quality of generative AI workflows that review large document sets and draft structured findings for expert reviewers in a regulated environment. You will write and tune the prompts, retrieval and output logic, and own how the program proves the outputs are accurate and well-evidenced enough for experts to rely on. What you'll do * Design and build generative AI workflows that review large document sets against defined criteria, identify potential issues, and draft structured findings and documents. * Build the evaluation framework: gold sets from historical reviews, and metrics for accuracy, evidence linkage, unsupported statements and consistency, run as a regression check before every prompt or model change. * Set prompt engineering and retrieval standards: chunking, retrieval over source documents and reference standards, and structured outputs that match application contracts. * Build document chunking and retrieval over large, mixed-format document sets. * Define provenance and confidence: prompt and model versioning in MLflow, a confidence score reviewers can act on, and links to the evidence behind each statement. * Measure improvement from reviewer edits by comparing AI drafts with the final human versions, and feed the results back into prompts. * Build monitoring of output quality and usage, and document each version and its results. * Set guardrails for confidential data, and explain results to business leaders in plain language. * At the Principal level, set standards across workflows and mentor other data scientists. What you bring * Senior: 8+ years in data science or machine learning. Principal: 12+ years. Both include 2+ years building production applications on large language models. * Python and PySpark; Databricks notebooks and MLflow. * Proven evaluation design for generated text: gold sets, automated scoring and human review loops. * Databricks Mosaic AI: Model Serving, Vector Search and Agent Framework. * Document understanding: extracting content from PDF, Excel and scanned files. * Ability to explain model behavior and limits to non-technical domain experts. * Experience with Azure OpenAI models. Nice to have * Audit, accounting or regulatory domain knowledge. * Model-graded evaluation and human-in-the-loop review systems. Working with us * You will work inside a secure, private cloud environment with highly confidential data. * You must be authorized to work in the United States and able to pass a background check. * We are looking for people who can start within a few weeks. Pay: $120,000.00 - $125,000.00 per year Benefits: * 401(k) * Dental insurance * Health insurance * Paid time off * Vision insurance Application Question(s): * Do you have Python and PySpark;Databricks notebooks and MLflow experience? * Do you have experience with Azure OpenAI models? Experience: * Data Science or Machine learning : 8 years (Required) Work Location: Hybrid remote in Ashburn, VA