XJTAG logo
Open search box button Shopping cart button

Your Development Board Cost Thousands. What Happens When It Stops Working?

Image of CEO of XJTAG Simon Payne Simon Payne, CEO, XJTAG |  Subscribe to LinkedIn newsletter
XJTAG Testing Times - PCB engineers by test bench

FPGA and SoC development platforms are powerful, complex and sometimes extremely expensive. Perhaps they deserve a test strategy of their own.

Development boards have changed dramatically.

What was once a relatively simple PCB containing an FPGA, some memory and a few connectors can now be an extremely sophisticated system in its own right.

Today’s FPGA and SoC development platforms may contain:

  • High-end FPGAs or adaptive SoCs
  • Multiple memory technologies
  • High-speed serial interfaces
  • Ethernet PHYs
  • PCIe
  • Power-management circuitry
  • ADCs and DACs
  • Configuration memories
  • Expansion connectors
  • Multiple voltage and power domains

They are invaluable engineering tools, they can also represent a significant investment.

So what happens when one suddenly stops behaving as expected?

The First Question Is Usually: What Changed?

Perhaps the board worked perfectly yesterday.

Now the FPGA won’t configure.

The processor won’t boot.

DDR initialisation fails.

An Ethernet interface has disappeared.

Or a mezzanine card that previously worked is no longer being detected.

The natural reaction is to look at whatever changed most recently.

Was it the FPGA image?

The software?

A configuration setting?

A cable?

The power supply?

The expansion card?

That is entirely reasonable.

But eventually another possibility has to be considered:

What if the development board itself has developed a hardware problem?

Development Boards Have Difficult Lives

Development platforms don’t always live in gentle environments. They sit on engineers’ benches, cables are connected and disconnected repeatedly. expansion cards are changed, Jumpers and switches are moved, and power is cycled constantly.

Signals may be brought onto the board from external circuitry that is itself still under development. And unlike the final product, the development platform may be used in configurations its designers could never reasonably have anticipated.

Most of the time, these boards tolerate this remarkably well.

But they are still electronic assemblies.

Components can fail.

Connectors can be damaged.

Solder joints can develop problems.

Power circuitry can be affected.

And accidental electrical overstress is always possible when prototype hardware is involved.

The Problem Is Establishing What You Know

When something stops working, the challenge isn’t necessarily repairing it immediately.

The first challenge is establishing where the problem actually is.

Suppose an FPGA no longer communicates with a peripheral.

Is the FPGA configuration wrong?

Has the peripheral failed?

Is there a problem with the PCB connection between them?

Is a power rail missing?

Has a clock disappeared?

Or is the application software simply no longer configuring the interface correctly?

The symptom alone rarely tells you.

And that can turn an apparently simple problem into hours—or days—of investigation.

Vendor Support Is Valuable, but Independent Evidence Is Valuable Too

Development-board manufacturers generally provide extensive documentation, reference designs, software examples and technical support.

All of that is extremely useful.

But there is still value in being able to independently establish confidence in the hardware sitting on your own bench.

Not because vendor support has failed.

And not because you should be expected to repair every problem yourself.

But because additional evidence can dramatically improve the engineering conversation.

Instead of saying:

“The board doesn’t seem to work anymore.”

you may be able to establish:

“The JTAG chain is intact, these connections test correctly, this device responds as expected, but this area of the board does not.”

That is a very different starting point.

A Development Board Is Still a PCB

It is easy to think of a development platform as a complete product.

Plug it in.

Download the tools.

Load the design.

Start developing.

But electrically, it is still a PCB full of interconnected semiconductor devices.

The FPGA or SoC at the centre of the board may provide access to a surprisingly large amount of surrounding circuitry.

That means the development board itself may already contain much of the hardware required to perform useful structural tests.

Boundary Scan Gives You Another View of the Hardware

A normal development workflow tends to look at the board through the application.

Configure the FPGA.

Boot the processor.

Run software.

Exercise the interface.

That works extremely well—until the application itself cannot run.

Boundary Scan provides a different route.

Because its test logic operates independently of the application, it may still be possible to access the hardware even when the FPGA design, processor software or operating system is unavailable.

Depending on the board and the devices fitted, Boundary Scan may help engineers investigate areas such as:

  • JTAG chain integrity
  • Connections between Boundary Scan devices
  • Memories
  • Peripheral interfaces
  • GPIO
  • Control signals
  • Configuration circuitry
  • Other accessible non-JTAG components

It doesn’t replace the vendor’s diagnostics or your normal development tools.

It provides another source of evidence.

Proving What Still Works

This is perhaps the most useful mindset.

When an expensive development board develops an unexplained problem, don’t begin by assuming that the entire board is faulty.

Start by asking: What can I prove still works?

Can you communicate with the devices in the JTAG chain?

Can you verify connections between major devices?

Can you exercise an attached memory?

Can you control or observe the signals connected to a peripheral?

Can you establish that a particular section of the PCB is behaving correctly?

Every positive result eliminates another possible cause.

And as we’ve discussed throughout Testing Times, reducing the list of possibilities is often the fastest route to understanding the real problem.

The Same Principle Applies to Your Own Prototype

There is a wider lesson here.

If independent structural testing is useful for an expensive commercial development platform, it is probably worth considering for the hardware your own engineering team is developing as well.

A development board has usually been designed by an experienced engineering team with extensive debug access.

Your first prototype may not have that luxury. Thinking about test access during your own design process can provide the same advantage:

the ability to establish confidence in the hardware independently of the application running on it. That becomes especially valuable when the software and hardware are being developed at the same time.

Perhaps Development Boards Deserve a Test Strategy Too

We spend a lot of time thinking about the products we design.

Far less attention is given to the development equipment we depend on to create them.

Yet a high-end FPGA or SoC development board may be one of the most important pieces of hardware on an engineer’s bench.

If losing access to it can stop development, having a way to establish whether the board itself is healthy starts to make sense.

Not necessarily as a full manufacturing test.

Not as a replacement for vendor support.

But as a practical engineering diagnostic.

Where Boundary Scan Testing Fits

This is one of the areas where Boundary Scan software such as XJTAG can be useful.

By combining the Boundary Scan capability already built into the devices with information about how the development board is connected, XJTAG can provide an independent way of exercising and checking accessible parts of the hardware.

For supported development platforms, that can turn the JTAG connection already present on the board into something more than a programming and debugging interface.

It becomes another route to understanding the PCB itself.

And importantly, it complements the manufacturer’s existing tools rather than replacing them.

Closing Thought

Modern development boards are extraordinarily capable pieces of engineering.

They are also becoming increasingly complex and increasingly valuable.

When one stops behaving as expected, the most useful question isn’t immediately:

“Is the board broken?”

It is: “What can I independently prove about the hardware?”

Vendor diagnostics, application software and conventional bench equipment all have an important role to play.

Boundary Scan can add another layer of evidence.

And when a development project is waiting for an answer, that additional visibility can be extremely valuable. Perhaps the expensive development board on your bench deserves a test strategy of its own.

Board Design Tip of the Week

The next time you select a development board, don’t look only at the FPGA, processor, memory capacity and interfaces.

Also look at the diagnostic access the platform provides.

Is the JTAG chain readily accessible?

Are schematics available?

Can important devices and interfaces be reached independently of the application?

Those details may seem relatively unimportant when everything is working.

They become considerably more valuable the day it isn’t.

Want to See What’s Possible on Your Development Board?

XJTAG supports a growing range of FPGA and SoC development and evaluation boards, helping engineers use the Boundary Scan capability already built into the hardware to gain greater visibility into the PCB.

If you’re using a supported platform, you can explore the available boards and see how XJTAG can help you test and diagnose the hardware here:

Explore XJTAG supported development and evaluation boards: https://www.xjtag.com/development-evaluation-boards/

Discover more about your board design with the Free XJTAG Layout Viewer.

To get started find out how the Free XJTAG Layout Viewer can help you see on your board:

Download your Free XJTAG Layout Viewer today

XJTAG Testing Times cartoon - Grounded

Related Reading

complex development board with memories

Find the fault. Save the board.

Start free trial
Testing Times - PCB engineers in workshop, asking what JTAG is
Your Device Has JTAG. But Can You Actually Use It? Many engineers select an FPGA, processor or SoC and see JTAG support listed in the device specification. That sounds reassuring. The capability is there. But there is an important catch: If the PCB design doesn't provide practical access to that JTAG interface, much of the value has already been lost before the first board is built. And that can mean giving up access to one of the most useful diagnostic capabilities already built into the silicon. JTAG Is Easy to Overlook During Design When a new PCB is being developed, there is a lot competing for attention. DDR, PCIe, Ethernet, Clocks, Power, High-speed interfaces and Peripherals. Compared with those, JTAG can look relatively unimportant. It may be seen simply as something required to program an FPGA or connect a processor debugger. And if programming is going to happen through another mechanism, it can be tempting to think: “We don't really need JTAG on the finished board.” That decision may remove considerably more capability than expected. JTAG Is More Than a Programming Interface Programming and processor debug are probably the most familiar uses of JTAG. But, as we've explored throughout Testing Times, many modern semiconductor devices also contain Boundary Scan logic. That can give engineers access to the signals around the device independently of the application software. With the right board design, that access may help verify PCB connectivity, investigate memories and peripherals, exercise control signals and establish confidence in hardware before the application is even running. The semiconductor manufacturer has already provided much of this capability inside the device. The PCB designer decides whether it remains accessible. The Capability Is Already There This is an important point. Adding JTAG access doesn't mean adding an entirely new test technology to the design. For many devices, the hardware is already in the silicon. The design task is largely about making sensible provision for using it. That might mean routing the JTAG signals to a connector, test pads or another accessible interface. On a more complex board, it may mean considering how several devices should be arranged into one or more JTAG chains. The incremental design effort can be small. The potential value later can be considerable. Think Beyond the First Prototype The obvious time to value JTAG access is during bring-up. If the processor doesn't boot or the FPGA application isn't ready, JTAG and Boundary Scan may provide another route into the hardware. But its usefulness doesn't necessarily stop once the prototype works. The same access can potentially support: Programming and configuration Development and debug Structural testing Manufacturing test Failure analysis Repair and rework Long-term product support A connector that seems unnecessary during schematic capture may become extremely useful several years later when someone has to diagnose a board that no longer behaves as expected. Don't Just Put the JTAG Pins on the Schematic Providing JTAG access requires a little more thought than simply connecting TCK, TMS, TDI and TDO. During design, it is worth asking: Where will the interface actually be accessed? Are all the devices you may want to reach included in a usable chain? What happens if one device is unpowered or not fitted? Are suitable pull resistors and termination options provided? Can relevant mode or configuration pins be placed into the required state? Would several JTAG chains provide better flexibility on a complex system? Will JTAG still be accessible on the production board? These are relatively easy questions to answer before layout. They become much harder to solve once assembled PCBs are sitting on the bench. What Happens If You Don't Route It Out? Perhaps nothing. The board may work perfectly. The firmware may work first time. Production may run smoothly. But if something does go wrong, one useful source of visibility is no longer available. A processor that won't boot becomes harder to investigate. An FPGA board with incomplete firmware may provide fewer diagnostic options. A production failure may need to be approached entirely through functional testing or physical probing. The problem isn't that the product suddenly cannot be tested. It is that an existing diagnostic option has been designed away. Closing Thought Seeing “JTAG supported” in the device datasheet isn't enough. The semiconductor manufacturer has provided the capability. The PCB designer decides whether anyone will actually be able to use it. Routing JTAG access may occupy only a very small part of the schematic. But it can provide useful visibility throughout the life of the product. And the easiest time to preserve that capability is before the PCB exists. Board Design Tip of the Week If you're providing JTAG access, build in a little flexibility rather than designing only for the configuration you expect to use today. Consider providing footprints for appropriate JTAG pull resistors and termination components, even if some are initially left unpopulated. Where device configuration or mode pins can affect JTAG operation, consider providing resistor options that allow those states to be changed if necessary. And if a permanent JTAG connector isn't desirable on the production PCB, consider retaining accessible test pads or another practical connection method. Those small provisions cost very little at the design stage, but they can give engineers considerably more options during bring-up, test and fault diagnosis later. Could JTAG boundary scan help you test your next design? Discover how boundary scan can help you improve test coverage, diagnose faults faster and simplify board testing: Find out more about JTAG boundary scan and how it can help you. Discover more about your board design with the XJTAG Layout Viewer. To get started find out how the XJTAG Layout Viewer can help you see on your board: Download your free XJTAG Layout Viewer today
Testing Times - PCB engineers in workshop, testing before firmware
Can You Test a Board Before the Firmware Exists? We were recently asked a very practical question: “Can you test our board using Boundary Scan? The firmware isn’t available yet.” The answer is yes, and this is exactly the kind of situation where Boundary Scan can provide real value. When the first assembled PCBs arrive, hardware engineers want to start establishing confidence in them immediately. Waiting for firmware before meaningful hardware testing can begin creates an unnecessary dependency between two development activities that are often progressing in parallel. The Hardware Often Arrives Before the Firmware This is a familiar situation on many development projects. The first assembled PCBs arrive. The hardware team wants to begin bring-up. But perhaps: The processor firmware is still under development The FPGA image is not yet available Device drivers have not been written The operating system is not running None of this necessarily indicates a problem with the project. Hardware and software development simply progress at different rates. The important question is whether hardware testing has to wait. Boundary Scan Doesn't Depend on Your Application One of the key advantages of Boundary Scan is that the test capability is built into the semiconductor devices themselves. It operates independently of the application software or FPGA design. So the processor does not need to have booted and the FPGA does not need to be configured with the product design before useful structural testing can begin. Boundary Scan can instead be used to gain access to PCB signals through the dedicated test logic already present in suitable devices. That access can help engineers verify things such as: PCB interconnections Address, data and control buses Connections to memories Connections to peripherals Resets, enables and other control signals Accessible interfaces and discrete circuitry The emphasis is primarily on establishing confidence in the connections between the devices and the PCB. Those connections can also be used to exercise part of a non-JTAG device's functionality, providing additional evidence that the surrounding circuitry is behaving correctly. Why That Matters During Bring-Up Suppose the firmware eventually arrives and a memory interface doesn't work. There are now several possibilities. Is there a firmware problem? Is the memory being initialised incorrectly? Is the FPGA or processor configuration wrong? Or is there a physical problem with one of the PCB connections? If structural tests have already established confidence in the relevant hardware, one major area of uncertainty has already been reduced. Instead of the hardware and firmware teams having to work out whose area contains the problem, they have independent evidence to help narrow the investigation. That can make bring-up considerably more systematic. Hardware and Firmware Can Progress in Parallel This is perhaps the broader benefit. Boundary Scan can help decouple initial hardware validation from firmware availability. While the software team continues developing the application, the hardware team can begin verifying accessible areas of the PCB. Then, when the firmware is ready, it starts from a stronger position. Some of the underlying hardware has already been independently checked. That does not remove the need for functional testing. It means functional testing begins with greater confidence in the hardware underneath it. It Helps the Firmware Team Too Firmware engineers are often among the first people to expose hardware problems because their software begins exercising interfaces that may never previously have been used. If a peripheral fails to respond, they can spend considerable time investigating code before discovering that the cause is actually a PCB connectivity problem, an incorrectly fitted component or another hardware issue. Providing the firmware team with boards that have already passed useful structural tests gives them a much better starting point. They can concentrate on developing and debugging the software without automatically becoming the first stage of hardware diagnosis. Start Building Confidence Earlier Functional testing usually needs enough of the system to be operational for the application to run. Boundary Scan works differently. It uses dedicated test logic within the semiconductor devices to provide access to the PCB independently of the application. That means useful hardware testing can begin much earlier in the development process. And when hardware and software teams are both working towards the same deadline, removing unnecessary dependencies between them can be extremely valuable. Closing Thought The question we were asked was: “Can you test the board even though the firmware isn't ready?” Yes. But perhaps the more useful question is: “Why wait for firmware before starting to build confidence in the hardware?” Hardware and firmware development often happen in parallel. The test strategy can too. Boundary Scan provides a way to start checking the PCB earlier, establish confidence in accessible areas of the hardware and reduce uncertainty before the application ever runs. Board Design Tip of the Week When planning a new design, ask what you could verify if no application software were available at all. Can important resets and enables be controlled? Can key interfaces and memory connections be accessed? Can critical signals be driven and observed? Is the JTAG chain readily accessible? Thinking about those questions during the design stage can give the hardware team useful diagnostic capability from the moment the first assembled PCB arrives — rather than from the moment the firmware does. Could boundary scan help you test your next design? Discover how boundary scan can help you improve test coverage, diagnose faults faster and simplify board testing: Find out how XJTAG can help you. Discover more about your board design with the Free XJTAG Layout Viewer. To get started find out how the Free XJTAG Layout Viewer can help you see on your board: Download your Free XJTAG Layout Viewer today
XJTAG Testing Times - PCB engineers in workshop
“It’s Only a Small Board Change…” What Could Possibly Go Wrong? We recently spoke to an engineer about a new revision of an existing PCB. The message was: “It’s only an iterative change.” The implication was understandable: the design changes are small, so the additional test work should also be small. And they may be absolutely right. But it raises an important question: How do you know? A small PCB revision does not automatically mean a small test impact. How Much Test Work Does a Small PCB Revision Really Create? The answer depends less on how many things changed and more on what those changes affect. A reset signal may have moved. A buffer may have been added. A memory device may have been replaced. A previously unused FPGA pin may now be connected. A power rail or enable sequence may have changed. Each change may be small on the schematic, but if an existing test depends on it, the impact can be much greater. Every test makes assumptions about the hardware: which devices are fitted, how signals are connected, what can be controlled and observed, and how devices behave during test. Change one of those assumptions and the existing test may need to change too. What If the Change Is at the Centre of the Board? Now consider replacing the FPGA or processor at the heart of the design. Perhaps the new device performs essentially the same function and most of the surrounding circuitry is unchanged. It may look like a straightforward substitution. But that device may interact with a significant proportion of the board and therefore affect many existing tests. For teams using Boundary Scan, there is another important consideration: Is the BSDL file correct for the exact device and package now fitted? Boundary Scan software relies on the BSDL file to describe the device's pins, Boundary Scan cells, instructions and test capability. So a change that appears on the design change list as: “FPGA changed from A to B” could potentially affect considerably more of the existing test strategy than the change list initially suggests. This is why test effort isn't proportional to the number of changes. It depends on how central those changes are to the tests already developed. Regression Testing Isn't Just for Software Software engineers are familiar with regression testing. A small code change doesn't mean rewriting the entire application, but it does mean understanding what else could have been affected. PCB test deserves the same approach. For every revision, ask: What changed? Which existing tests depend on it? Which assumptions are still valid? Which tests can remain unchanged? Which tests need modifying? Is any additional test coverage required? The objective isn't to assume everything has changed. It's to avoid assuming nothing has. Reuse Should Still Be the Goal Last week, we looked at the value of reusing tests developed during bring-up through production, repair and support. A new board revision shouldn't change that principle. Good reusable tests should make iterative development easier. The aim is to preserve everything that remains valid and concentrate engineering effort only on the tests affected by the new design. Reuse the test — but validate the assumptions behind it. That is far more efficient than designing a complete test strategy again for every PCB revision. Closing Thought “It’s only an iterative change” may be completely true. And the additional test-development work may genuinely be trivial. But that should be the conclusion of an impact review, not the assumption at the start of one. When the next PCB revision is released, ask: What changed? Which of our existing tests depend on those changes? Are the assumptions behind those tests still valid? Then preserve everything that still works. The objective isn't to design a complete test from scratch. It's to know which tests need changing — and why. Board Design Tip of the Week When documenting your next PCB revision, add one more item to the change list: Test impact. For each design change, consider whether it affects: Component functionality or pinout Signal routing or connectivity Resets, enables or mode settings Power rails or sequencing Interfaces and control signals Physical test access Programming or configuration Existing test assumptions And if Boundary Scan forms part of your test strategy, also check whether the change affects the BSDL/device model or JTAG architecture. A short review at this stage can prevent a “small change” from becoming an unexpected test-development problem when the revised boards arrive. Discover more about your board design with the Free XJTAG Layout Viewer. To get started find out how the Free XJTAG Layout Viewer can help you see on your board: Download your Free XJTAG Layout Viewer today