1 - Stanford University



Exploring a CPLD/FPGA-based Triggering System for LCLS

Matthew T. Brown

Office of Science, Science Undergraduate Laboratory Internship Program

Georgia Institute of Technology, Atlanta

Stanford Linear Accelerator Center

Stanford, California

August 15, 2008

Prepared in partial fulfillment of the requirement of the Office of Science, Department of Energy’s Science Undergraduate Laboratory Internship under the direction of Ron Akre in the Klystron Test Laboratory at Stanford Linear Accelerator Center.

Participant: ___________________________________

Signature

Research Advisor: ___________________________________

Signature

Table of Contents

ABSTRACT………………………………………………………………………………………i

INTRODUCTION………………………………………………………………………………..1

METHODS AND MATERIALS………………………………………………………………....3

RESULTS………………………………………………………………………………………....5

DISCUSSION AND CONCLUSION………………………………………………………….....7

ACKNOWLEDGEMENTS……………………………………………………………………….8

REFERENCES……………………….………………………………….……………………....10

FIGURES AND TABLES……………………………………………………………………….11

ABSTRACT

Exploring a CPLD/FPGA-based Triggering System for LCLS. MATTHEW T. BROWN (Georgia Institute of Technology, Atlanta, GA 30332) RON A. AKRE (Stanford Linear Accelerator Center, Menlo Park, CA 94025).

The Klystron Department at the Stanford Linear Accelerator Center (SLAC) requires a new, more accurate triggering system in order to run tests involving the new Linac Coherent Light Source (LCLS) project. In order to meet requirements involving number of channels per test station, trigger repetition rate, delay length, pulse width, and most importantly phase jitter, complex programmable logic devices (CPLDs) and field-programmable gate arrays (FPGAs) were investigated and tested for use in such a triggering system. Schematic-based and VHDL-based design entry both were used in programming the logic devices with functions simulating that which will be used in the test lab. The CPLD that was tested proved too small to contain the amount of logic required for a single triggering station, but the FPGA that was tested was more than adequate. Jitter was measured and calculated to be within the picosecond range as specified by the test lab. The Arcturus Coldfire processor may be used to allow communication between the FPGA and a networked computer, enabling one to adjust the various trigger delays and widths once the FPGA has already been programmed. The FPGA also possesses digital clock managers (DCMs) which may be used to finely control the phase and timing of signals at a sub-clock cycle level. It is for these reasons that an FPGA-based triggering system was recommended for use in the Klystron Department test lab.

1. INTRODUCTION

With the advent of the Linac Coherent Light Source (LCLS), the Stanford Linear Accelerator Center (SLAC) has many facilities in need of upgrading. The linear accelerator (linac) itself is one of the most important of these, as it is being refitted and modified to drive the LCLS instead of serve as a particle collider. More stringent timing requirements are needed for the LCLS project than were required for previous experiments, and a new triggering system must be developed. In order to determine the best way to implement the new triggering system, a model triggering system was devised and tested.

The last kilometer of the linac will be used to accelerate bunches of electrons that will be passed through undulators, generating coherent hard x-ray radiation as their acceleration changes. In order to provide these electrons with the optimal energy, precise timing of the klystrons, modulators, and other radio frequency (RF) systems is paramount. The linac consists of about 240 klystrons, which supply energy to the beam line, in 30 sectors plus an injection sector. A three and one-eights inch copper coaxial cable runs along the length of the linac and serves as the main drive line (MDL) [1: 2]. A reference RF signal (476MHz) is supplied to the MDL, and, at each sector, the signal is passed through a x6 multiplier (increasing frequency to 2856MHz) to a klystron acting as a sub-booster, which drives the RF systems in the sector [2: 2]. The signal is then passed to the phase reference line (PRL), which is used as a stable phase reference source to monitor the phase of the RF for its sector [3: 1]. Phase and amplitude detectors (PADs) are also used to determine the state of the signal’s phase and amplitude between sectors [2: 6].

Piggy-backed on top of the reference RF signal in the MDL is a 360Hz signal that serves as a timing pulse, or a fiducial signal. The fiducial is simply a part of the reference signal that is somehow different from the rest of the main signal (a different amplitude, for instance), and thus it is detectable at each sector of the linac. Systems at each sector detect this fiducial and produce a 119MHz reference signal and a 360Hz clock signal. This signal is then passed to circuitry that detects it and sends out delayed pulses that act as triggers for the equipment along the linac [4: 1]. In other words, the triggering system detects the fiducial pulse and sets the appropriate delays for the equipment it monitors so that the equipment will fire at precisely the correct time.

The triggering system currently in place served previous projects well, but the equipment is in need of upgrades. A new triggering system using CPLDs (complex programmable logic devices) and FPGAs (field-programmable gate arrays) was tested to determine if it could meet the requirements for parameters such as phase jitter, pulse repetition rate, pulse delay, and pulse amplitude. Of particular importance was minimizing the phase jitter. Jitter, usually negligible, appears in the time domain as a signal “jumping” back and forth and in the frequency domain as a band of frequencies around the desired one. An ideal signal contains no jitter; however, consecutive pulses in real signals occur shortly before or shortly after they are expected due to abnormalities in the environment and equipment [2: 3]. The accuracy required by the triggering system dictated that the jitter ideally be on the order of picoseconds. FPGAs and CPLDs required testing in order to determine if this goal was feasible.

The test lab for which this system is being proposed consists of thirteen stations with six to eight trigger channels needed per station. At the stations with eight triggers, six of the triggers will be at the standard transistor-transistor logic (TTL) level of 5V, and the other two will be 50V triggers. The length of delays required is on the order of 20 μs. Some of the triggers will need to have an adjustable width, while some will be fixed at a 1 μs width. Additionally, the ability to make sub-clock cycle adjustments to phase will be required, so the use of a delay locked loop (DLL) may be necessary.

2. METHODS AND MATERIALS

Several pieces of hardware and software were required to perform the tests on the CPLD and FPGA. The tested hardware included a Xilinx Cool Runner XPLA XCR3064XL CPLD on a previously-fabricated circuit board with a USB interface, four Canadian Bank (BNC) input ports, and four BNC output ports. This CPLD contains 1500 logic gates that the programmer can configure to perform a variety of functions. The FPGA used was a Xilinx Spartan 3 XC3S200PQ208-5, and it was mounted on an LCLS PAC (phase and amplitude control) board. This FPGA contains 200,000 logic gates, allowing for much more complex designs. The PAC board included a number of interfaces (Ethernet, COM port) that could be used to communicate with the onboard Arcturus Coldfire processor. Two BNC output ports and two SMA input trigger ports (and a clock input port) were used to communicate with the FPGA.

The piece of software used to program these devices was the Xilinx ISE WebPack v.10.1.02. Initially, the schematic editor was used to program the gate arrays, but later VHDL dominated the majority of the coding. The iMPACT simulation program was used in conjunction with the Xilinx design software to simulate test signals prior to programming the boards, and the more powerful stand-alone program ModelSimXE also was used for this purpose later in the design.

Several pieces of hardware were used in testing these programmable chips. For a quick probe of the output signals, the Tektronix TDS 3052 oscilloscope was used. For a slightly more accurate and sensitive look at the phase jitter, the Tektronix TDS 684B was employed for the analysis. To generate simulated NIM pulses (a simulated fiducial with ‘0’ at 0V and ‘1’ at -.8V), the Stanford Research Systems (SRS) Model DS345 30 MHz Synthesized Function Generator was used.

To measure phase jitter of the CPLD on the Tektronix TDS 684B, both the simulated input fiducial and the output pulse generated from it were connected to the oscilloscope via BNC cables. The oscilloscope then was programmed to trigger off the input channel. Magnifying the input and output waveforms many times and setting the sampling preferences to “Envelope (100)” produced thick lines of the signals (due to the oscilloscope holding 100 different samples of the waveforms on the screen at once instead of a single sample of each). In this way, the width of the two waveforms’ lines (determined in part by the jitter present) was measured in picoseconds and a comparison was made. If the output waveform were much thicker than the input, significant phase jitter may have occurred. It should be noted that some jitter associated with the internal circuitry of the oscilloscope itself will always exist, and it should be taken into account.

However, in order to more accurately measure the root mean square (rms) phase jitter of the FPGA, a slightly more controlled setup was required. A low-noise 102 MHz clock was connected to the external clock input of the FPGA board, and a simple divide-by-four function was implemented on the FPGA itself. The output, a 25.5 MHz signal, was passed through a low-pass filter to get rid of harmonics (producing a pure signal) and into a PAD board, which digitized the analog signal and passed the digital signal to a computer. Software was run on the computer which could measure the standard deviation of phase noise of the output signal, which was converted to time domain jitter through the equation 2πft = φ, where f was 25.5MHz, φ the phase noise values in radians, and t the time domain jitter in seconds.

In the coding of the CPLD, schematic design entry was used as the top-level design. A simple twenty-four bit counter was triggered by a simulated fiducial input and used to count clock cycles until a specific value, set by a comparator, was reached, at which point the output pulse was driven high. After a certain number of clock cycles, which was set by another comparator, the flip flop passing the output was cleared and the output went low again. Four different output channels were generated in this way, requiring sixteen comparators (four per channel) and four flip flops for each of the outputs.

However, when programming the FPGA, a text-based approach was used, as it was deemed simpler than managing several sheets of schematic. VHDL was used to program essentially the same functionality as the CPLD in the FPGA. The simulated NIM pulse entered one of the input ports, which triggered the initialization of the counter. A start and stop time for the output pulse was specified for each output channel desired. Upon reaching these values, the FPGA drove the output high (start time) and then low (stop time), generating the delayed trigger pulses.

3. RESULTS

Figure 1 depicts the device summary after programming the Cool Runner XPLA XCR3064XL CPLD with one output trigger channel and a 24-bit counter (which, when clocked at 119MHz, generates sufficient delay between the incoming fiducial and the output pulse). However, as highlighted, 63% of the macrocells and 40% of the registers have already been consumed by this simple test. If another channel and registers (something as simple as a D type flip flop) were added to store the start and stop times, the CPLD would run out of gates.

The Spartan 3 XC3S200PQ208-5 FPGA’s device summary after programming a 24-bit counter and two output trigger channels can be seen in Figure 2. As indicated in the Figure, the test consumed only 1% of the device’s flip flops and look-up tables (LUTs). In addition, the program only used 2% of the slices on the FPGA itself.

The results of the testing of the CPLD for phase jitter on the Tektronix TDS 684B can be seen in Figures 3, 4, and 5. Figure 3 shows a wide view of what was measured – the clock signal in green and the simulated output pulse, delayed a few cycles, in white. The oscilloscope was triggered off the clock signal, whose jitter, highlighted in Figure 4, was measured to be about 260 ps. Figure 5 indicates the approximate jitter of the output trigger pulse was measured to be 264 ps, a difference so small that one cannot distinguish the jitter added by the CPLD from the jitter added by the internal circuitry of the oscilloscope.

A more accurate measurement of jitter was obtained for the FPGA, and a plot of one such measurement can be seen in Figure 6. The measurement of greatest interest made by the computer program for this test is the standard deviation of the phase, highlighted in the Figure, which is measured for this trial over a ten second interval. The standard deviations of ten such ten second intervals were recorded and can be seen in Table 1. The phase standard deviations were converted to time domain jitter measurements and averaged, resulting in the average value of about 1.99 ps as shown in Table 1.

In order to obtain a baseline reference for this jitter measurement, the same experimental procedure was then repeated for the PAD board itself. A clean 25.5MHz signal was fed into the PAD board, and the jitter of the output was measured. Table 2, similar to Table 1, shows the average jitter of ten trials of measuring the standard deviation of the low noise circuit.

4. DISCUSSION AND CONCLUSION

Figure 1 clearly implies that the Cool Runner XPLA XCR3064XL CPLD was insufficient for this triggering system. The device simply did not have enough gates to support eight different output channels plus the overhead of an interface with a computer complete with many more registers. However, Figure 2 indicates that the FPGA tested showed great promise and would suit the test lab triggering system well. Since practically only 1% of the device was used with two output channels in operation, virtually no danger of using all the gates on the chip exists.

The 24-bit counter, incrementing on a 119MHz clock, was enough to generate the delays required by the test lab. Similarly, this counter/comparator configuration was enough to generate 1 μs trigger pulses, and by changing the start and stop times through an interface with a computer, variable pulse lengths can be achieved.

By simply using a divide-by-four configuration on the FPGA to test the jitter, an idea of the best possible jitter was obtained. The 2 ps measurement is low enough jitter to meet the test lab requirements, so with careful programming of the FPGA, a jitter value close to this measurement should be attainable.

The Spartan-3 series of FPGAs has four global digital clock managers (DCMs) onboard. These blocks contain a DLL architecture which can be used to control the phase of an outgoing clock signal. The underlying principle of a DLL is the input clock signal, CLKIN, is compared to a feedback clock, CLKFB, and adjustments may be made to the phase of the output clocks of the DCM based on this comparison. The DCMs on the Spartan-3 FPGA have the ability to finely adjust the phase with either a fixed value or a variable value that may be changed by the application loaded onto the FPGA. In order to use this fine adjustment feature, the frequency mode variable of the DLL on the DCM must be set to the low frequency option, which limits the input clock frequency range to 18 MHz - 167 MHz [5: 77]. The minimum phase shift increment is the larger of either 1/256th of the clock period or a board-specific value between 30ps and 60ps [5: 118]. Figure 7 depicts the interaction of the four inputs of the phase shift operation, PSCLK, PSEN, PSINCDEC, and PSDONE [5:121]. PSCLK clocks the phase shift change itself, PSEN enables the change, and PSINCDEC determines whether to increment (‘1’) or decrement (‘0’) the phase. By using this phase shifter and DLL combination, one can reduce clock skew on an outgoing signal by altering the phase of a critical signal at a sub-clock cycle level. It should also be noted that each DCM has nine outputs (an output with twice the input clock frequency, an output shifted 180○, etc.), and fine phase adjustment affects each of the nine outputs in the same fashion [5:115].

The CPLD proved insufficient in the qualities necessary to merit a proposal for the test lab. The FPGA, however, met all of the requirements provided by the test lab and should be used when preparing circuit boards for the lab. More than enough logic gates are on the chip to provide the necessary amount of channels, delay lengths, and pulse widths. The presence of DCMs allows for fine timing adjustments at the sub-clock cycle scale, although the chip tested only had four such blocks. Phase jitter imposed on output signals by the FPGA is negligible, as well, which is key in providing an accurate timing system.

5. ACKNOWLEDGEMENTS

This work was completed in the summer of 2008 and supported by the U.S. Department of Energy, Office of Science and the Stanford Linear Accelerator Center. I would like to thank my mentor Ron Akre for providing guidance and keeping me on the right track through the internship. I would also like to thank Bo Hong and Anatoly Krasnykh for guiding me through the testing of the logic devices and Jeff Olsen for assisting me with the VHDL programming aspect of the project. Finally, I would like to acknowledge Steve Rock, Susan Schultz, and Farah Rahbar for organizing and managing the SULI program at SLAC.

6. REFERENCES

[1] H. D. Schwartz. “Computer Control of RF at SLAC,” presented at the 1985 Particle Accelerator Conference, Vancouver, B.C, Canada, 1985.

[2] R. K. Jobe, “Phase Conventions for the SLC,” [SLC.DOC.SPEC]PHASE.TEX. Stanford, CA, 1986.

[3] J. D. Fox and H. D. Schwartz. “Phase and Amplitude Detection System for the Stanford Linear Accelerator,” contributed to the Particle Accelerator Conference, Santa Fe, New Mexico, 1983.

[4] P. Krejcik, et al, “Timing and Synchronization at LCLS,” SLAC, Melno Park, 2007.

[5] “Spartan-3 Generation FPGA User Guide,” UG331 (v1.4) 25 June, 2008. Accessed 1 August 2008. .

7. FIGURES AND TABLES

................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download