Senior Principal Technical Architect – AI, Data Platforms & Cyber Security
Job Description
Company: Reuben Cooley Inc.
Location: New York metropolitan area, US
Senior Principal Technical Architect – AI, Data Platforms & Cyber Security
Location- Princeton, NJ & NYC, NY (Hybrid)
Experience Level: 15+ Years (Executive / Principal Level)
Role Overview:
We are seeking a visionary and hands-on Principal Technical Architect to lead the architecture, design, security, and strategic evolution of our Enterprise Data Platforms and Multi-Agent GenAI Systems. In this role, you will bridge the gap between complex enterprise data engineering, modern cloud architecture, cutting-edge Generative AI applications, and enterprise cybersecurity controls.
You will design zero-touch automated data observability solutions, multi-agent AI pipelines, zero-trust data access patterns, and enterprise-wide GenAI adoption frameworks. The ideal candidate brings a deep technical background in Databricks, cloud platforms, Python, contract-driven LLM architectures, threat modeling for AI systems, and proven enterprise leadership in scaling and securing AI solutions across the organization.
Key Responsibilities:
1. AI Systems & Multi-Agent Architecture
Architect Multi-Agent AI Pipelines: Design end-to-end, LLM-powered multi-agent frameworks (deterministic + generative) using Pydantic contracts, asynchronous Python, and provider-agnostic model integration (e.g., OpenAI SDK, Databricks Model Serving).
AI Tooling & Scaffolding: Build graph-based execution builders, dynamic YAML rules engines, capability registry patterns, and structured diagnostic retry mechanisms for LLM agents.
Human-in-the-Loop Integration: Implement validation and curation layers that enable user DAG editing, schema validation, and error repair before scaffold generation.
2. AI Security, Risk & Guardrails
LLM Threat Modeling & Abuse Prevention: Lead abuse-case modeling, prompt injection defense, jailbreak mitigation, and red-teaming strategies for LLM agents and RAG architectures.
Output Guardrails & Data Protection: Implement payload masking, custom SQL validation layers, row-count caps, and automated PII/PCI detection to prevent data exfiltration via AI interfaces.
Identity & Access Governance: Architect hybrid identity flows (OAuth 2.0, Okta/Entra ID), Service Principal access patterns, and automated token lifecycle/rotation management (e.g., Delta Sharing tokens).
3. Enterprise Data Platforms & Observability
Databricks Estate Architecture: Lead large-scale data platform migrations, estate auto-discovery, and governance automation across Databricks workspaces (Unity Catalog, Workflows, Jobs API, Delta Lake, Delta Sharing).
Data Governance & Zero-Trust Access: Enforce fine-grained authorization models including Row-Level Security (RLS), dynamic column masking, and centralized data classification in Unity Catalog.
Data Quality & Observability Frameworks: Design automated, multi-tiered data quality verification platforms capable of real-time incident detection, automated table onboarding, and continuous file freshness tracking.
Platform Governance & FinOps: Oversee multi-cloud cost governance (AWS, Azure, GCP), resource optimization, and infrastructure governance to maximize ROI while maintaining compliance.
4. Enterprise GenAI Adoption & Governance
Org-Wide Transformation: Define and execute adoption strategies for developer AI tooling (e.g., GitHub Copilot, custom LLM assistants) and establish measurement frameworks for code quality, productivity gains, and ROI.
Enablement & Standards: Conduct technical workshops, build best-practices documentation, create reusable architectural patterns, and mentor engineering teams across divisions.
Compliance & Security Auditing: Oversee access recertification, SIEM logging/auditing mechanisms, and regulatory compliance (e.g., regional data residency and vendor risk governance).
Source: LinkedIn
