AprilTags in FRC: How Robots See the Field

AprilTags are the black-and-white square markers you see mounted around an FRC field, and they work like a QR code that your robot can read from a distance. When a camera on your robot spots one, the robot can figure out exactly where it is standing on the field, because the location of every tag is published before the season starts.

That single capability, knowing your own position accurately, is what makes modern autonomous routines, auto-aiming, and driver-assist alignment possible. Here is how the whole system works, from the pattern on the wall to the pose your code actually uses.

What an AprilTag actually is

An AprilTag is a fiducial marker: a printed pattern designed to be detected quickly and identified reliably by a camera. They were developed by the APRIL robotics lab at the University of Michigan and they long predate FRC, since robotics researchers have used them for years to track objects and localize robots.

Each tag encodes a single number, its ID. The grid of black and white squares inside the border is not decoration; it is a binary code with built-in error correction, so a camera can read the ID correctly even at an angle or in mediocre lighting. FRC uses the 36h11 tag family (it switched to 36h11 for the 2024 season, replacing the older 16h5 family used in 2023, the first year AprilTags appeared on the FRC field). Every tag on the field has a unique ID, usually numbered 1 through however many the game uses that year.

The tags are printed at a fixed, known size, roughly 6.5 inches (about 16.5 cm) on the black square, and that known size is what makes the geometry work: because your code knows the tag's real-world dimensions, it can reason backwards from how big and skewed the tag looks in the image to how far away and at what angle the camera must be. Confirm the exact size against the current field drawings, and for the full detail on tag families and the coprocessors that read them, see our lesson on AprilTags and vision coprocessors.

Where the tags live on the field

Every AprilTag on an FRC field sits at a surveyed, fixed location, and FIRST publishes those locations. WPILib ships this as an AprilTagFieldLayout, effectively a lookup table mapping each tag ID to its exact position and orientation. The number of tags and where they sit changes every year with the game, so always pull the current field drawings and layout file from the game manual rather than assuming last year's map.

Those positions live in the standard WPILib field coordinate system, the same frame every pose your vision system produces lives in, so getting comfortable with it pays off fast; our lesson on the field coordinate system and odometry walks through how WPILib defines field position with Pose2d and Rotation2d. If terms like pose, fiducial, or field-relative are new to you, our glossary defines the vocabulary you will hit constantly here.

How a robot turns a tag into a position

The path from "a camera sees a square" to "my robot is at (4.2, 1.8) facing 30 degrees" runs through three steps, and it is worth understanding the shape of it even when a library does the heavy lifting:

1. Detection. The vision software scans each camera frame, finds any AprilTags, decodes their IDs, and locates the four corners of each tag in the image with sub-pixel precision.
2. Pose solving (PnP). From those four corners, the tag's known size, and the camera's calibration, the software solves the Perspective-n-Point problem to get the tag's 3D pose relative to the camera: how far away it is and how it is rotated.
3. Transform to the field. Combine that with where the tag sits on the field (from the AprilTagFieldLayout) and where the camera is mounted on your robot, and you have your robot's pose on the field.

Step two only works if the camera is calibrated to its exact lens and resolution; an uncalibrated pipeline produces poses that look plausible but are quietly off by feet, which is one of the most common reasons a team's estimates come out wrong. PhotonVision and Limelight both ship calibration routines, and our lesson on turning tag detections into a field-relative pose covers the detection-to-pose path in code.

Pose estimation: fusing vision with odometry

Here is the part that trips people up: you almost never use the raw vision pose by itself. Vision measurements are accurate but arrive intermittently, can be noisy, and lag by a few tens of milliseconds while the coprocessor works. Your drivetrain encoders and gyro, meanwhile, give a smooth, high-rate estimate of motion that slowly drifts over a match. Pose estimation fuses the two so you get the best of both: smooth, high-rate, and drift-free.

WPILib does this out of the box with a pose estimator that runs on your odometry and lets you fold in each vision measurement, tagged with the frame's timestamp so the correction lands at the right moment in its history, which is how it compensates for camera latency. You also tell it how much to trust vision versus odometry: trust vision too much and your position snaps around every time a tag flickers in and out of view; trust it too little and you never correct odometry drift. That tuning, and the code around it, is where a lot of the real engineering happens, our lessons on fusing vision with odometry and tuning trust with standard deviations work through the whole pipeline.

The tools you will actually use

You do not implement tag detection from scratch. Two solutions dominate FRC. PhotonVision is free, open-source software you flash onto a small coprocessor, an Orange Pi, a Raspberry Pi, or similar, paired with a USB camera, with strong WPILib integration and a lot of control. Limelight is a self-contained smart camera, camera, coprocessor, and LEDs in one box, that costs more but is close to plug-and-play. Both run AprilTag pipelines, both publish poses over NetworkTables, and both are used by top teams; the pick comes down to your budget and how much you want to tinker versus buy a mostly finished tool. If you are weighing the two head to head on features, latency, and cost, our getting-started AprilTag vision walkthrough gets a PhotonVision coprocessor reading tags end to end.

One hardware detail matters a lot: use a global-shutter camera. A rolling-shutter camera captures the image one row at a time, so a moving robot smears the tag and wrecks the pose solution; a global shutter (built into the Limelight; the Arducam OV9281 is a popular PhotonVision choice) captures the whole frame at once and stays sharp while you drive. Mount the camera rigidly, too, because a bracket that flexes quietly throws off the transform to your robot. That intersects with clean power and wiring, which our Electrical & Wiring guides cover.

Multi-tag solving

A single tag gives you a usable pose, but it can be ambiguous: from certain angles the math has two nearly-equal solutions and can flip between them. Seeing two or more tags at once fixes this. Both major platforms support multi-tag solving, PhotonVision through a multi-tag PnP strategy and Limelight through its MegaTag2 approach, which also folds in your gyro heading to lock down orientation. Whenever your camera can frame more than one tag the resulting pose is far more stable, which is exactly why teams think carefully about camera placement, and why our lesson on multi-tag fusion treats it as a core part of getting to centimeter-level localization.

What teams actually use this for

The point of all this is not to know your coordinates for their own sake. Accurate pose unlocks a few things that win matches:

• Auto-aiming and alignment. The robot drives itself to a precise scoring position, or lines up a shot, without the driver eyeballing it. That is faster and more repeatable under pressure, and our vision-aligned scoring mini-project builds exactly that: an AprilTag aim controller fused with pose estimation to drive up and score.
• Better autonomous. Instead of dead-reckoning off encoders alone and slowly drifting, an auton routine can correct its position off tags mid-run and hit its marks reliably across different fields and battery states.
• Driver assist. Even in teleop, a button that snaps the robot to a known alignment saves a second or two every cycle, and seconds add up over a match.

Teams that align quickly and score consistently tend to rank higher and get picked, which is why vision quietly became a scouting talking point too.

Gotchas that bite rookie programmers

A few hard-won lessons save weeks of frustration:

• Calibrate the camera. Do the calibration for your exact camera and resolution, or your poses will be quietly off by feet.
• Reject bad measurements. Filter out poses from far-away single tags, high-ambiguity detections, and anything that places you off the field. Feeding garbage into the pose estimator is worse than feeding it nothing.
• Get the timestamps right. Latency compensation only works if you pass the true capture time. The vision libraries hand you a proper timestamp; use it instead of the current clock time.
• Mind your coordinate frames. A large share of vision bugs come down to a sign flip or a swapped axis in the camera-to-robot transform. Measure the mounting position carefully and test against a known spot on the field.

Where to start

If you are the one setting this up, the realistic path is short: pick PhotonVision or Limelight, get a global-shutter camera calibrated, confirm you can see a tag and read a pose in the web dashboard, then feed that pose into a WPILib pose estimator and drive around watching your position on a field widget. Get that loop working on a bench before you ever try to auto-align at an event.

For the programming fundamentals underneath all of this, our Become a Robot Programmer path and the Programming, Controls & Sensors guides walk through the control system, sensors, and command-based structure that AprilTag code plugs into. Vision feels like magic the first time your robot locks onto a target on its own, but under the hood it is just a camera, a known map, and some careful geometry.