Choose LLMs, RAG, MCP, tool calling, evaluation, agents, or production reliability. Get the project idea, implementation steps, deliverables, and checks for that skill only.
Build one dependable feature around a hosted or local language model, with a strict output contract and a human review before any reply is sent.
Project scenario
A support team receives short tickets and needs a suggested category, urgency, and draft reply. The tool assists the agent; it never sends messages or changes customer records.
Build only this
One request/response endpoint, a fixed ticket schema, a model adapter, validated structured output, and a small test set.
Keep out of scope
No autonomous sending, customer database integration, RAG, agent loop, or fine-tuning. Keep this project about the LLM call and its contract.
Tools for this project
Python or TypeScriptone model APIJSON Schema or typed validationunit testsenvironment variables
Create 20 synthetic tickets. Define an output schema with category enum, urgency enum, short rationale, and reply draft. Decide which fields may be uncertain and what invalid/unsafe requests should return. Write a deterministic keyword baseline for category first.
Produce
Ticket fixtures, versioned output schema, baseline classifier, and a short product boundary document.
Verify before moving on
The baseline runs locally; schema rejects unknown categories, missing required fields, and oversized output; no real customer data or secrets are committed.
Intermediate
Connect the system
Turn the working idea into a reliable, testable flow.
I1
Add one provider behind an adapter
Implement
Create a `generateTicketSuggestion(input, config)` boundary. Put the API key in an environment variable, set request timeout and output-token limits, and keep provider-specific request/response mapping inside the adapter. Do not leak the key or raw ticket into logs.
Produce
A small model adapter with typed input/output and a mock implementation for tests.
Verify before moving on
Tests run with the mock and no key; missing credentials and provider timeouts produce a safe, actionable error rather than crashing the application.
Advanced
Hardening and proof
Improve reliability, evidence, and production judgment.
A1
Constrain generation and validate twice
Implement
Send only the ticket fields the feature needs. Ask for the defined structured response, parse it, validate against the schema, and reject or repair only through a bounded retry. Keep the original ticket and generated draft separate; never treat model text as a command.
Produce
Prompt/template version, structured model response path, validation errors, and human-review UI or CLI output.
Verify before moving on
Malformed JSON, invalid enum values, empty replies, and prompt-injection text in the ticket cannot bypass output validation or trigger an external action.
A2
Measure usefulness, latency, and cost
Implement
Run the same held-out fixtures through the keyword baseline and model. Record category accuracy, urgency errors, schema-valid rate, latency, and estimated cost per ticket. Review false urgent labels and inappropriate reply drafts manually.
Produce
A reproducible evaluation script, comparison table, error examples, and a release note with limitations.
Verify before moving on
You can explain where the model beats the baseline, where it fails, and why a support agent must approve every draft.
01 / Pick a direction
Six paths. Different kinds of impact.
You do not have to choose forever. Start with the work you want to do most days, then build the skills that prove you can do it.
Career-fit check-in
What kind of work would you like to do?
Pick up to three activities that sound energizing. This creates a preference-based starting suggestion, not an aptitude score or hiring prediction.
How to interpret these suggestions
The order reflects your selected work preferences, not a score computed from your background. Compare the first two paths by reading their daily work, skill gaps, and project evidence; try a small task from each before committing.
Calibrate your starting point
Start where you are.
Choose the description that feels closest. It changes the first advice, not your ceiling.
Start with one programming language, Git, and small complete programs. Learn SQL and basic statistics alongside it; do not begin by trying to train a large model.
First move for this background
Complete one small program end-to-end before adding an AI dependency. Keep the code, tests, and README together.
Watch for
Do not confuse copying a notebook or prompt with being able to debug, test, and maintain the system.
First deliverable for AI application engineer
Create a small service that accepts structured input, validates it, stores it, and exposes a documented API.
Show: A clean repository, setup instructions, tests, and a deployed demo.
02 / Your selected roadmap
AI application engineer
Build useful products around models: assistants, search, extraction, recommendations, and workflows that connect AI to real software.
Milestones checked off0%
Progress is saved on this device.
Good fit if
People who like building user-facing software and iterating quickly.
Typical work
Design APIs and interfaces, connect models and tools, evaluate outputs, and make the whole feature reliable in production.
Tools to explore
Python or TypeScriptmodel APIsSQLretrievaltestingcloud deployment
01
Programming, data, and web-service fundamentals
Become a dependable software builder
Learn
Python fundamentals, functions, modules, and tests
Git, HTTP, JSON, SQL, and API design
One backend framework and enough frontend to ship a usable interface
Build
Create a small service that accepts structured input, validates it, stores it, and exposes a documented API.
Show your work
A clean repository, setup instructions, tests, and a deployed demo.
Mastery gate
A second developer can run the service; invalid input and dependency failure have explicit, tested behavior.
02
Model selection, prompting, structured output, and failure analysis
Learn model behavior and evaluation
Learn
Tokens, context limits, latency, and inference cost
Embeddings and when retrieval is better than fine-tuning
A test set, task-specific metrics, and human review for ambiguous cases
Build
Add a model-powered capability to the service and compare at least two approaches against a small labeled test set.
Show your work
A results table with examples of successes, failures, and the decision you made from them.
Mastery gate
A held-out task set compares a baseline and the AI approach; quality, latency, and cost trade-offs are written down.
03
Retrieval, tool use, orchestration, and graceful failure
Make the feature robust
Learn
Chunking, metadata filters, retrieval quality, and citations
Retries, timeouts, rate limits, caching, and deterministic fallbacks
Prompt-injection risks, access control, privacy, and data retention
Build
Build a source-grounded assistant over a bounded document set with citations, refusal behavior, and an evaluation suite.
Show your work
A threat model, retrieval tests, and clear behavior when the source does not contain an answer.
Mastery gate
Retrieved claims link to source evidence; prompt injection, permissions, abstention, and empty retrieval have regression tests.
04
Production operations and product-quality feedback loops
Ship, observe, and improve
Learn
Tracing, structured logs, user feedback, and quality monitoring
Deployment, secrets management, and safe schema changes
Cost and latency budgets plus a plan for regressions
Build
Deploy the complete feature and run a documented release review using real or carefully simulated usage.
Show your work
A live demo, architecture note, evaluation report, and a short walkthrough of trade-offs.
Mastery gate
A release has traces, quality and cost signals, a rollback route, and an owner for reviewing user feedback.
Self-assessment / not a test
Find the next skill gap to close.
Rate what you can do today. Role targets are working guides, not hiring bars; change a rating as your evidence grows.
Self-ratings are private to this browser.
For AI application engineer
Suggested next areas
Programming & softwareLevel 1 of 3
Systems & operationsLevel 1 of 3
Evaluation & riskLevel 1 of 3
Statistics & mathLevel 1 of 2
Build evidence with projects; a self-rating alone does not establish proficiency.
How to rate yourself reliably
Choose the highest level you can demonstrate more than once without a step-by-step tutorial. Use a repository, experiment report, deployment, or peer review as evidence; lower the rating when an important part still depends on guesswork.
03 / Skills that travel
Build foundations, then specialize.
Strong practitioners know enough of the whole system to spot weak assumptions and ask better questions.
A
Programming & data
Write readable code, use Git, test changes, work with APIs, and query structured data. Python is broadly useful; TypeScript is valuable for product-facing AI.
Practice + evidence
Practice: Build a small, tested API or analysis tool; review a pull request and fix an issue from a clean checkout.
Evidence: A reproducible repository with tests, dependency instructions, and readable commit history.
B
Math with a purpose
Build probability, statistics, vectors, and optimization as your chosen work requires. Learn enough to reason about assumptions and metrics, not to collect certificates.
Practice + evidence
Practice: Work through a metric or experiment by hand, then verify the calculation in code and explain its assumptions.
Evidence: A notebook or memo showing a baseline, uncertainty, and why the metric matches the decision.
C
Model literacy
Understand training versus inference, data and evaluation, generalization, uncertainty, and where a simpler non-AI solution is better.
Practice + evidence
Practice: Compare an AI approach with a deterministic baseline on representative and adversarial examples.
Evidence: A test set, failure analysis, and a clear explanation of when not to use the model.
D
Engineering practice
Use reproducible environments, versioned data and code, sensible access controls, observability, and clear documentation.
Practice + evidence
Practice: Deploy a small service; add structured logs, an input contract, a health check, and a rollback note.
Evidence: Another person can operate the service and diagnose a deliberately simulated failure.
E
Responsible judgment
Consider privacy, security, accessibility, bias, consent, human oversight, and the cost of errors before deployment.
Practice + evidence
Practice: Threat-model one feature and examine privacy, accessibility, disparate impacts, misuse, and human escalation.
Evidence: A risk register maps each significant risk to a test, mitigation, owner, and residual-risk note.
Possible domain specializations
Language & searchComputer visionSpeech & audioRoboticsRecommendationsAI securityHealthcareClimate & science
Pick a domain after you have a working foundation. Learn its data, users, regulations, and failure costs; a domain-aware engineer is more useful than a tool collector.
A short, reviewed resource shelf
Study the next thing you need.
Start with a small selection tied to ai application engineer. Time is an estimate for selected material, not a provider guarantee.
Hugging FaceFree resource
Large Language Models Course
Work through transformers, datasets, fine-tuning, and sharing models.
Prerequisites
Good Python; introductory deep learning helps.
Effort
Course guidance: 6-8 hours per chapter; begin with chapters 1-4.
Add a model-powered capability to the service and compare at least two approaches against a small labeled test set.
Link checked 26 Sep 2026. Recheck content and terms before planning.
Weekly study loop / 4-6 hours
Turn reading into proof.
Study one concept and reproduce a small example.
Build a project slice with a test or baseline.
Review evidence, write a short note, and ask for feedback.
04 / Make the portfolio count
Two strong projects beat ten thin demos.
Build for one real user and make your reasoning visible. Choose a brief for ai application engineer.
PROJECT 01
Evidence-first knowledge assistant
Answer questions from a small, public document collection. Include source links, abstain when evidence is missing, and let reviewers flag weak answers.
Credible evidence
Publish retrieval and answer-quality results, failure examples, and a basic privacy and prompt-injection review.
PROJECT 02
Structured intake and triage workflow
Turn messy requests into a validated schema, route them by explicit rules, and keep a human approval step for consequential actions.
Credible evidence
Show a baseline, schema validation, safe failure behavior, and a short demo from input to review.
PROJECT 03Advanced / production systems project
Production RAG support-operations copilot
Build an internal assistant for a fictional multi-region support organization. It searches versioned product manuals, approved troubleshooting runbooks, and policy documents; respects team and region access; cites the exact source passages; asks a clarifying question when the request is underspecified; and hands off unresolved or high-risk cases to a human with a compact evidence bundle.
Credible evidence
Deliver an architecture diagram, ingestion and deletion pipeline, retrieval benchmark, answer evaluation report, authorization tests, adversarial test suite, latency/cost report, operations runbook, and a recorded demo using synthetic or explicitly public documents.
Advanced execution / AI application engineer
Build the project like a small production engagement.
Problem framing and deterministic baselinesWrite three realistic user questions, the allowed document set, one out-of-scope request, and what the product should do when evidence is missing. Implement a simple keyword or FAQ baseline first.
Retrieval and grounded generationAdd document ingestion, metadata, retrieval, and an answer path. Keep source IDs in the response. Compare retrieved results with the baseline on the same questions.
Adversarial testing and application securityAdd tests for prompt injection in source text, cross-user access, empty retrieval, malformed input, model timeout, and unsafe output. Fix and retest each failure.
Observability and product operationsInstrument request ID, latency, retrieval outcome, errors, and estimated model cost without logging sensitive content. Add a release checklist and rollback notes.
Hands-on learning loop
Build Lab: learn by making the work.
Evidence-first knowledge assistant: Answer questions from a small, public document collection. Include source links, abstain when evidence is missing, and let reviewers flag weak answers.
Repeat this loop at every stageStudy one idea → build a small artifact → test it against the acceptance check → write what changed your understanding.
01
LEARN THIS CONCEPT
Problem framing and deterministic baselines
Build task
Write three realistic user questions, the allowed document set, one out-of-scope request, and what the product should do when evidence is missing. Implement a simple keyword or FAQ baseline first.
Produce
A one-page product brief, a small versioned question set, and a working baseline.
Acceptance check
A reviewer can identify the user, success condition, allowed sources, and safe fallback without reading your code.
02
LEARN THIS CONCEPT
Retrieval and grounded generation
Build task
Add document ingestion, metadata, retrieval, and an answer path. Keep source IDs in the response. Compare retrieved results with the baseline on the same questions.
Produce
A runnable assistant with source references and a result table for the fixed question set.
Acceptance check
Every supported answer points to retrieved evidence; missing evidence returns a clear abstention rather than an invented answer.
03
LEARN THIS CONCEPT
Adversarial testing and application security
Build task
Add tests for prompt injection in source text, cross-user access, empty retrieval, malformed input, model timeout, and unsafe output. Fix and retest each failure.
Produce
A regression suite, a small threat model, and before/after failure notes.
Acceptance check
The tests fail before the fix, pass afterward, and do not expose content outside the caller's allowed sources.
04
LEARN THIS CONCEPT
Observability and product operations
Build task
Instrument request ID, latency, retrieval outcome, errors, and estimated model cost without logging sensitive content. Add a release checklist and rollback notes.
Produce
A deployed or locally runnable release candidate with an operations note and evaluation report.
Acceptance check
A teammate can diagnose a failed request, inspect quality/cost signals, and roll back without exposing secrets or user data.
Every project should answer these questions
Who is it for, what decision or task does it improve, and what is deliberately out of scope?
What baseline did you compare against, and how did you measure quality, latency, and cost?
What failed? How does the system handle uncertainty, bad inputs, privacy, and misuse?
Can another person run it, inspect the evidence, and understand your design choices?
Portfolio quality check
Review one project before you publish.
0 of 6 review points addressed
Checklist progress is saved on this device; checked items are reminders, not a quality certification.
05 / Turn skill into opportunity
Show evidence. Tell a precise story.
Hiring expectations vary by team and seniority. Use job descriptions as a local signal, then tailor your proof to the work they actually need.
Search with several titles
Start with: AI engineer · Generative AI engineer · Applied AI engineer · Software engineer, AI products.
Read 15 to 20 current role descriptions
Track repeated skills, level expectations, domain knowledge, and the evidence requested. Ignore tools that appear only once unless a target employer requires them.
Write outcome-first case studies
State the problem, your contribution, the baseline, measured result, limitations, and what you would improve next.
Practice explaining trade-offs
Be ready to discuss evaluation, data leakage, failure handling, privacy, latency, and why you chose this approach over a simpler one.
Keep a weekly learning loop
Study one concept, build or test one small thing, write down what broke, and get feedback from another person.
Progression signals / not a time promise
What stronger ownership looks like
Entry evidence
Ship a bounded AI feature with tests, baseline evaluation, and safe failure behavior.
Next-level signal
Own quality, latency, cost, privacy, and product outcomes across more than one release.
Adjacent moves
Applied ML, AI platform, evaluation engineering, or technical product engineering.
Progress by increasing scope, reliability, and the quality of your decisions, not by collecting tools or relying on a fixed number of years.
Personal application workbench
Turn research into better answers.
Draft privately, compare real listings in your target market, and use the recurring evidence to guide your next learning milestone.
Interview practice / AI application engineer
Practice explaining your decisions.
Use a project you actually built. Include assumptions, evidence, trade-offs, and what you would change.
Use this answer structure
Context: who needed what, and what constraint made the problem hard?
Decision: what baseline and alternatives did you consider, and why this approach?
Evidence: what did you measure, on what data, and what changed after review?
Limits: what failed, what risks remain, and what would you do next?
Review each answer forA concrete example, your own contribution, a measurement, one trade-off, and an honest limitation.
Drafts are stored only in this browser. Do not enter confidential information.Opportunity research / live search
Check what employers ask for near you.
Search current listings yourself; this guide does not scrape, verify, or summarize job posts or salaries.
Build one dependable feature around a hosted or local language model, with a strict output contract and a human review before any reply is sent.
Define the contract before the prompt
Implement: Create 20 synthetic tickets. Define an output schema with category enum, urgency enum, short rationale, and reply draft. Decide which fields may be uncertain and what invalid/unsafe requests should return. Write a deterministic keyword baseline for category first.
Produce: Ticket fixtures, versioned output schema, baseline classifier, and a short product boundary document.
Verify: The baseline runs locally; schema rejects unknown categories, missing required fields, and oversized output; no real customer data or secrets are committed.
Add one provider behind an adapter
Implement: Create a `generateTicketSuggestion(input, config)` boundary. Put the API key in an environment variable, set request timeout and output-token limits, and keep provider-specific request/response mapping inside the adapter. Do not leak the key or raw ticket into logs.
Produce: A small model adapter with typed input/output and a mock implementation for tests.
Verify: Tests run with the mock and no key; missing credentials and provider timeouts produce a safe, actionable error rather than crashing the application.
Constrain generation and validate twice
Implement: Send only the ticket fields the feature needs. Ask for the defined structured response, parse it, validate against the schema, and reject or repair only through a bounded retry. Keep the original ticket and generated draft separate; never treat model text as a command.
Produce: Prompt/template version, structured model response path, validation errors, and human-review UI or CLI output.
Verify: Malformed JSON, invalid enum values, empty replies, and prompt-injection text in the ticket cannot bypass output validation or trigger an external action.
Measure usefulness, latency, and cost
Implement: Run the same held-out fixtures through the keyword baseline and model. Record category accuracy, urgency errors, schema-valid rate, latency, and estimated cost per ticket. Review false urgent labels and inappropriate reply drafts manually.
Produce: A reproducible evaluation script, comparison table, error examples, and a release note with limitations.
Verify: You can explain where the model beats the baseline, where it fails, and why a support agent must approve every draft.
Career roadmap / AI application engineer
Build useful products around models: assistants, search, extraction, recommendations, and workflows that connect AI to real software.
Roadmap
Become a dependable software builder Create a small service that accepts structured input, validates it, stores it, and exposes a documented API.
Learn model behavior and evaluation Add a model-powered capability to the service and compare at least two approaches against a small labeled test set.
Make the feature robust Build a source-grounded assistant over a bounded document set with citations, refusal behavior, and an evaluation suite.
Ship, observe, and improve Deploy the complete feature and run a documented release review using real or carefully simulated usage.
Systems & operations: self-rated 1, guide target 3
Evaluation & risk: self-rated 1, guide target 3
Statistics & math: self-rated 1, guide target 2
Portfolio briefs
Evidence-first knowledge assistant: Answer questions from a small, public document collection. Include source links, abstain when evidence is missing, and let reviewers flag weak answers.
Structured intake and triage workflow: Turn messy requests into a validated schema, route them by explicit rules, and keep a human approval step for consequential actions.
Production RAG support-operations copilot: Build an internal assistant for a fictional multi-region support organization. It searches versioned product manuals, approved troubleshooting runbooks, and policy documents; respects team and region access; cites the exact source passages; asks a clarifying question when the request is underspecified; and hands off unresolved or high-risk cases to a human with a compact evidence bundle.
Build Lab: Evidence-first knowledge assistant
Answer questions from a small, public document collection. Include source links, abstain when evidence is missing, and let reviewers flag weak answers.
Problem framing and deterministic baselines
Build: Write three realistic user questions, the allowed document set, one out-of-scope request, and what the product should do when evidence is missing. Implement a simple keyword or FAQ baseline first.
Deliver: A one-page product brief, a small versioned question set, and a working baseline.
Verify: A reviewer can identify the user, success condition, allowed sources, and safe fallback without reading your code.
Retrieval and grounded generation
Build: Add document ingestion, metadata, retrieval, and an answer path. Keep source IDs in the response. Compare retrieved results with the baseline on the same questions.
Deliver: A runnable assistant with source references and a result table for the fixed question set.
Verify: Every supported answer points to retrieved evidence; missing evidence returns a clear abstention rather than an invented answer.
Adversarial testing and application security
Build: Add tests for prompt injection in source text, cross-user access, empty retrieval, malformed input, model timeout, and unsafe output. Fix and retest each failure.
Deliver: A regression suite, a small threat model, and before/after failure notes.
Verify: The tests fail before the fix, pass afterward, and do not expose content outside the caller's allowed sources.
Observability and product operations
Build: Instrument request ID, latency, retrieval outcome, errors, and estimated model cost without logging sensitive content. Add a release checklist and rollback notes.
Deliver: A deployed or locally runnable release candidate with an operations note and evaluation report.
Verify: A teammate can diagnose a failed request, inspect quality/cost signals, and roll back without exposing secrets or user data.
Personal notes and progress are not included. Targets are guides, not hiring criteria.
This guide is a practical starting point, not a hiring guarantee. Role names and requirements differ across organizations; verify current expectations in your target market.