What Computer Architects Do

Computer architects decide how a processor is organized, so that real programs run fast enough within a budget of silicon area, power and design time. They don't draw transistors or write most of the RTL. They choose the pipeline, the caches, the branch predictor and hundreds of smaller features, and they back each choice with numbers before anyone commits to building it. This article explains what that job looks like in industry, how one decision is made, what numbers architects watch, and who they work with.

12 min read | Beginner

What You'll Learn

• Describe what computer architects do in a chip project
• Follow the loop from workload to model to decision
• List the numbers architects track and why
• Explain who architects work with, and when in a project

Architecture Is a Set of Decisions

A processor is the result of thousands of decisions. How many pipeline stages? How large a cache? Is it worth adding a second ALU? Each has a cost in area, power, frequency and engineering time, and each interacts with the others. No formula gives the answer, because the right choice depends on what the chip will run and how much it may cost.

A computer architect is the person responsible for those decisions. The word covers several related jobs. An instruction set architect defines the contract with software, as What an ISA Defines explains, a job few people do. A microarchitect defines how the hardware implements it, and most architects are this. A performance architect builds models and studies workloads. A system architect looks at the whole chip: cores, memory, interconnect and accelerators, as described in the SoC Design tutorial.

The work is less about clever invention than about judgment. Most of the good ideas are already known, and many appear in this tutorial. The hard part is choosing which ones this product needs, and which ones it does not.

The Architect's Loop

Architects work in a loop, which repeats from the first sketch until the design is frozen:

1. Start from workloads and targets. What will the chip run, and how fast must it be? Workloads and Benchmarks explains why everything else depends on this.
2. Propose a change. Perhaps a larger branch predictor or a new instruction.
3. Model it. Use a spreadsheet, a trace model or a simulator, as in Performance Modeling.
4. Estimate the cost in area, power, timing and effort.
5. Decide. Compare the gain with the cost, using the method in PPA Trade-offs and Amdahl's Law.
6. Check against reality. As RTL and layout data arrive, update the model, and repeat.

Most ideas fail at step 3 or 5, and that is the point. It's far cheaper to reject an idea in a spreadsheet than in silicon.

One Decision, From End to End

Here is a real example. A simple five-stage core stalls whenever an instruction needs a result from the one just before it. Should it have forwarding, a set of extra paths that pass results straight to the next instruction? Data Hazards and Forwarding explains the circuit.

The question is answered in steps. Workloads first: six small programs covering loops, sorting, checksums and pointers. Then a measurement: the same design is built with and without the feature, and each program is run. Forwarding gave a speedup between about 1.17 and 1.7. Then the cost: the area rose by about 2.5 percent. There was no effect on the instruction set, the compiler or software, because only the microarchitecture changed.

The decision is easy: a gain of 17 to 70 percent for 2.5 percent of area. Most decisions are much closer. A 3 percent gain for 3 percent more area, with a timing risk, can go either way, and it's the sort of thing architects argue about in meetings, with data on the table.

The Numbers Architects Track

Architects live by a small set of numbers. Each has come up in this tutorial:

The last line is often the deciding one. A feature that adds a large verification risk, for a small gain, may be dropped however good it looks, because a late chip earns nothing. Good architects have a keen sense of which ideas are easy to specify, but hard to get right.

Where Architects Fit in a Project

Architects are most active at the start, but they stay involved. A typical project looks like this:

• Definition. Marketing and customers define the product. Architects run trade-off studies, and propose what's feasible.
• Microarchitecture. They write the spec, fix the pipeline and sizes, and set PPA budgets.
• RTL and verification. They answer questions, review the design, and check that performance targets are still being met.
• Implementation. When timing or power runs short, they decide what to give up.
• Silicon. They measure the real chip against the model, and feed lessons into the next design.

A core design typically takes from one to several years. Since a design started today ships years from now, architects are always predicting: what will software look like when this chip arrives, and what will competitors have?

Who Architects Work With

Almost nothing an architect does stands alone. They work with RTL designers, who know what fits in a cycle; verification engineers, who know what's hard to check, as Verifying a CPU describes; physical designers, who know what fits on the die; compiler and software teams, who know what software really does; and product managers, who know what customers will pay for.

The ability to explain a trade-off to all these people is at least as important as technical depth. An architect who can't say why a decision was made, in a sentence with numbers, won't be able to get it through a review.

How to Think Like an Architect

A few habits separate good architects from those who simply know a lot about processors:

• Quantify. Say "12 percent on these workloads", rather than "much faster".
• Find the bottleneck first. Use a CPI stack to see where cycles go, before thinking of fixes.
• Think about the common case, but make sure that the rare case works.
• Know the physics. Know what takes a cycle, what costs area, and what uses power.
• Look for simple solutions. A cheap trick that gets 80 percent of the gain often beats a complex one that gets 100 percent.
• Remember the software. A feature that compilers can't use is wasted.

What a Week Looks Like

A typical week mixes many kinds of work. An architect might spend Monday reading the results of an overnight batch of simulations, and find that a change meant to speed up one workload has made another one worse. On Tuesday, they read through traces to find out why, and discover that a queue is filling up. On Wednesday, a physical design engineer reports that a path in the load pipeline misses timing by 40 picoseconds, and asks whether one cycle of latency can be added. The architect runs the model with the extra cycle, finds that it costs 1.5 percent on key workloads, and goes back with two alternatives.

On Thursday, there is a review of a colleague's section of the spec, looking for corner cases. On Friday, a customer asks whether a new instruction would help their code, which means reading the code, estimating the gain, and writing a short note. Little of this is drawing block diagrams. Most of it is reading data, asking why, and writing down decisions clearly enough that others can check them.

The hardest part is usually the lack of a clean answer. The data is incomplete, the workloads are stand-ins for software that doesn't exist yet, and the deadline is fixed. Good architects still make a clear decision, state their assumptions, and revisit it when better data arrives.

Becoming an Architect

Few people start as architects. Most begin in RTL design, verification or performance modeling, and move across after a few years, once they know how real designs behave. A solid grasp of the material in this tutorial is the starting point. Beyond that, it's worth building something: a small simulator, a cache model, or a simple core. Reading published descriptions of real processors, and trying to work out why they were built that way, is excellent practice too. Computer Architecture Interview Questions gives a sense of what interviews expect.

It also helps to learn a little of every neighboring field: how a compiler schedules code, how timing closure works, how a verification team thinks. Architects spend their days trading between those areas.

Common Mistakes