Basics of AI Engineering

Before we let AI write a single line of Cypress code, we need to understand the foundations of how these tools work. Knowing the basics of AI engineering is what separates frustrating, unpredictable results from fast, high-quality ones.

This lesson is free - it is the foundation everything else in the course builds upon.

What is an LLM?

An LLM (Large Language Model) is an AI model trained on vast amounts of text (and code) that predicts the most likely continuation of the input it receives. Tools like Claude, used by Claude Code, are powered by LLMs.

👨‍🏫 An LLM does not "know" your project. It only knows what you put in front of it - your prompt, your files, and the conversation you have with it.

Prompts

A prompt is the instruction you give the model. The clearer and more specific your prompt, the better the result.

A good prompt for testing usually includes:

• Context - what app, what behavior, what file.
• Goal - what you want (e.g., "write an E2E test for the login flow").
• Constraints - selectors strategy, style, framework version.

👨‍🏫 Compare "write a test" with "write a Cypress E2E test that visits the sample app, fills the form, submits it, and asserts the success message - use data-testid selectors." The second prompt produces far better results than the first.

Tools

Modern AI assistants can use tools - actions the model can take beyond generating text, such as reading files, running commands, or searching the web. This is what turns a chatbot into a coding agent.

Agents and agent modes

An agent is an AI assistant that can plan and take multiple steps - reading files, editing code, running tests - to accomplish a task autonomously.

Most AI coding tools expose different agent modes that constrain how the assistant behaves:

• Ask - the assistant answers questions and explains, without editing files.
• Plan - the assistant researches and proposes a step-by-step plan for your approval before making changes.
• Edit - the assistant makes code changes directly (often with your approval per change).

👨‍🏫 Use plan mode for non-trivial work so you can review the approach before any code is written, and ask mode when you just want an explanation. Reserve edit mode for when you are confident about the change.

Skills

A skill is a reusable, packaged set of instructions and knowledge that specializes the assistant for a task. The official Cypress skills (cypress-author, cypress-explain, cypress-docs) are exactly this - they make the assistant much better at Cypress work. You will install and use them in the next lessons.

👨‍🏫 Skills are also token-efficient. The assistant only loads a short piece of metadata (a name and description) for each installed skill up front, and reads the full skill file on-demand - only when the task actually calls for it. That way many skills can be available without filling up the context window.

Hallucinations

LLMs can generate output that sounds confident and plausible but is factually wrong. This is called a hallucination - the model invents selectors that do not exist, references API methods that are not real, or describes behavior the app does not have.

👨‍🏫 This is not a bug you can fix by rephrasing the prompt. It is a fundamental property of how LLMs work. The only reliable safeguard is to always run the generated code and review it critically before committing.

In the context of test automation, a hallucinated test is one that passes for the wrong reason or fails because it targets elements that do not exist. Skills like cypress-author reduce hallucinations by grounding the assistant in Cypress-specific knowledge, but they do not eliminate them entirely - your review is always the last line of defense.

Hallucinations also become more likely when the context window (covered just below) fills up. As a conversation grows long and older information gets summarized or dropped, the model loses track of details and starts filling the gaps with plausible-sounding inventions. Keeping conversations focused and starting fresh for unrelated tasks helps reduce this risk.

Context window

The context window is the maximum amount of information - measured in tokens - the model can consider at once. It includes your prompt, the files you share, and the conversation history.

👨‍🏫 When the context window fills up, older information may be summarized or dropped. Keep conversations focused, and start fresh for unrelated tasks.

Token consumption

Tokens are the chunks of text the model reads and generates (roughly word fragments). Both your input and the model's output consume tokens, which determines cost and counts against the context window.

👨‍🏫 Sharing an entire repository when you only need one file wastes tokens and dilutes the model's focus. Be deliberate about what you include.

This is also why skills are designed to be token-efficient. Rather than loading every skill's full instructions into the context, the assistant only keeps a small piece of metadata - the skill's name and description - in context for each installed skill. It reads the complete skill file only on-demand, when the task at hand actually requires it. This keeps the context window free for the work that matters, even when many skills are installed.

Suggested content 📚 and documentation links 🔗

• Introducing Claude Code
• Prompt engineering overview - Anthropic docs
• Cypress AI toolkit
• 75% Less Cypress Bloat in Your Agent's Context Window

Exercise 🎯

Pick any AI coding tool you have access to (Claude Code, Cursor, or GitHub Copilot) and practice the difference a good prompt makes.

1. Ask it a vague prompt: "write a Cypress test".
2. Then ask a specific one: "write a Cypress E2E test that visits https://tat-csc.s3.sa-east-1.amazonaws.com/index.html, and asserts the page loaded successfully - use resilient selectors and the AAA pattern".
3. Compare the two outputs and note the difference.

👨‍🏫 Also try switching between ask, plan, and edit modes for the same request and observe how the assistant's behavior changes.

The specific prompt should produce a runnable test with a clear describe/it structure and a meaningful assertion, while the vague one will likely guess at the app.

Show the world what you learned 🌎

To show your professional network what you have learned in this lesson, post the following on LinkedIn.

I started the "Cypress Testing in the AI Era" course by @Walmyr Lima e Silva Filho at the @Talking About Testing School, where I learned the basics of AI engineering for test automation - LLMs, prompts, tools, agents, agent modes, skills, context windows, token consumption, and hallucinations. #TalkingAboutTesting #TATSchool #CypressTestingInTheAIEra #Cypress #AI

👨‍🏫 Remember to tag me in your post. Here is my LinkedIn profile.