AI & Decision Support Engineer with Security Clearance

Job Description

Company: Johns Hopkins University Applied Physics Laboratory

Location: Laurel, US

Description The Systems Performance Analysis Group (KBS) at the Johns Hopkins University Applied Physics Laboratory is seeking a Generative AI & Decision Support Engineer to design, build, and deploy AI-enabled analytical and decision-support applications for Naval and Air Force sponsors. You will lead the development of human-LLM interaction capabilities, Retrieval-Augmented Generation (RAG) systems, and intelligent agents that operate at scale to support a variety of applications. This role will collaborate closely with internal stakeholders and external sponsors, including DoD organizations, to demonstrate and transition cutting-edge AI technologies into operationally relevant environments.

As an AI & Decision Support Engineer, you will… Architect, develop, and evolve a Generative AI-powered analytical applications that support human-LLM interactions and LLM-driven decision-making at scale.

Design and implement LLM agents that can autonomously plan, reason, and act using tools such as RAG, analytical tools, and relevant databases.

Apply prompt engineering techniques with state-of-the-art models to produce data-driven recommendations based operational and developmental data for a variety of systems.

Design, implement, and manage RAG pipelines using vector databases and frameworks to integrate relevant documentation, prior analytical results, and large information repositories into LLM decision-making.

Develop and maintain agents capable of: (1) Interpreting database schemas and generating SQL queries to answer user questions (SQL agents). (2) Translating natural language inputs into structured actions and game events stored in a database for downstream simulation and adjudication.

Experiment with and evaluate prompting strategies to improve reasoning quality, robustness, and transparency of model outputs in high-consequence decision contexts.

Integrate orchestration and observability tools (e.g., Prefect) to monitor LLM pipelines, track outputs, and provide real-time insight into system behavior during wargame execution.

Fine-tune and adapt foundation models (e.g., Llama, Mistral) using AWS SageMaker, Hugging Face TRL, and synthetic data generation (e.g., GPT-4 series) to optimize performance on sponsor-specific tasks such as deductive coding, domain knowledge transfer, etc.

Engage with internal leadership and external sponsors to demonstrate capabilities, collect requirements, and potentially transition systems to operational users.

Document system architectures, experiments, evaluation results, and operational guidance; prepare and present technical briefings and reports to technical and non-technical stakeholders.

Collaborate in multidisciplinary teams (AI/ML, software, human factors, operations analysts) to integrate AI capabilities into broader analytic and operational workflows.

Qualifications You meet our minimum qualifications for the job if you have…

Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, Data Science, or a closely related field.

Experience building production or prototype applications involving large language models (LLMs) or other generative models.

Proficiency in Python and at least one modern web/backend framework (e.g., FastAPI, Flask, Django).

Experience with relational databases, including schema design and query development (e.g., PostgreSQL).

Hands-on experience with at least one of:

• Retrieval-Augmented Generation (RAG) systems.
• LLM agent frameworks (tool use, ReAct, chain-of-thought-style prompting).
• Vector databases (e.g., Qdrant, ChromaDB).

Demonstrated ability to comprehend and synthesize complex technical or scientific information and make timely, well-reasoned decisions Strong written and oral communication skills, including experience preparing technical analyses and presenting findings to a range of audiences.

Hold an active Secret security clearance and can ultimately obtain Top Secret level clearance. If selected, you will be subject to a government security clearance investigation and must meet the requirements for access to classified information. Eligibility requirements include U.S. citizenship. You’ll go above and beyond our minimum requirements if you have…

Advanced degree (M.S. or Ph.D.) in Computer Science, AI/ML, Applied Mathematics, or related field.

Experience with

LLM frameworks and libraries (e.g., Hugging Face Transformers, TRL, LangChain).

Fine-tuning and evaluating open-weight models (e.g., Llama, Mistral) on domain-specific tasks.

Cloud platforms and MLOps tooling (e.g., AWS, SageMaker, Prefect, MLflow).

Frontend development using React and i

Source: BeBee