ClusterAlg.gif (577 bytes) The ATLAS Level-1 Calorimeter Trigger

[Home] [Architecture] [Meetings] [Participants] [Publications] [Software] [Search]

ATLAS-UK Level-1 Calorimeter Trigger Meeting

Thursday 23 January 2003 at RAL

Present: Cano Ay, Bruce Barnett, Christian Bird, Ian Brawn, Adam Davis, James Edwards, Eric Eisenhandler (chair), John Garvey, Norman Gee, Attila Hidvegi, Stephen Hillier, Murrough Landon, Gilles Mahout, Tamsin Moye, Viraj Perera, David Sankey, Uli Schäfer, Richard Staley, Thomas Trafzger, Peter Watkins


Click this side                               Click this side
for summaries                                for slides (pdf)
Jet/Energy Module hardware and tests....................Uli
CMM, ROD, TCM, etc. hardware and firmware status......Viraj
Cluster Processor Module hardware...................Richard
Cluster Processor Module tests.......................Gilles
Updated schedule.......................................Eric
Online software
JEM software and simulation..........................Thomas
Software summary...................................Murrough
Recent meeting highlights
Trigger/DAQ Steering Group.............................Eric
Any other business
Date of next UK meeting

NOTE: As Mainz and Stockholm people were in the UK for tests of the Jet/Energy Module, this UK meeting included reports on their progress but was otherwise abbreviated in order not to take too much time.


Jet/Energy Module hardware and tests - Uli Schäfer (slides)

Uli started with the JEM0 status. Currently one JEM (0.0) is available for standalone tests at Mainz. Due to connector misalignment it won’t fit in a crate. Two JEMs (0.1 and 0.2) are fully populated and standalone-tested prototypes currently under test at RAL. JEM0.2 has increased logic capacity in the main processor FPGA but is otherwise identical to the previous module. Firmware is lacking jet integration and RoI readout but is otherwise considered near-final. The jet-code speed problem is resolved. Andrea and Attila continue work on the firmware.

The final iteration, JEM1, is currently under design. Uli showed a block diagram and pointed out how the use of daughter modules allows solution of PCB routing and via density problems. He showed a list of modifications required to produce the final JEM modules.

Uli also showed drawings of the input daughter modules and the daughter connectors chosen. They are controlled differential impedance types. Finally, the status of JEM1 design activities was reported.

First results of JEM0 tests at RAL were presented. Both JEMs are up and running, and have been successfully operated after some earlier problems with a short on a JEM0.1 backplane connector and a bent pin. The LVDS receivers do not currently lock to TTC-driven LVDS sources and TTC jitter is suspected to be the cause of the problem. The issue will be further investigated, and once it is resolved (which it was later the same day - editor), tests on the input synchronisation and on the timing of the FIO links could start.

In discussion, Norman asked whether anyone knew how close we are to the limit of jitter in the TTC.

CMM, ROD, TCM, etc. hardware and firmware status - Viraj Perera (slides)

CP and Serialiser FPGA firmware: The firmware designs were demonstrated to interested parties on the 16th of January. No problems with the Serialiser firmware have been found so far. However, there is a problem in the CP firmware with the clock calibration logic (160 MHz). The multiplexed four-clock-phase solution does not work due to 'uncontrollable' skew in the clock distribution network. A two-phase solution is being tried currently and the initial results are positive. One possible solution to this problem is to distribute the four phases of the 160 MHz clock on the global nets and multiplex the data. However the Virtex-E (current) does not have sufficient global resources to try this solution on the existing CPMs, but a design can be targeted to a Virtex-II FPGA soon for evaluation.

Prototype ROD: Two modules have been tested with the CP slice data and CP RoI data firmware. CMM cluster, JEM slice and JEM RoI firmware awaits testing, as well as six more ROD modules. Neil Falconer from the ESS group will be helping with these tests. Specifications for the CMM jet and energy data are required to start design on the firmware.

CMM: The CMM has gone through various tests in the R25 lab with the use of DSS, GIO, RTM, TTC, CPM emulator and the ROD module. VME access, real-time path, readout data, connections to/from backplane, RTM, and front panel (CTP) have been checked. Also the CMM was tested with TTCrx dec non-rad-hard and rad-hard versions. The on-board FLASH can be programmed via the VME, and the FPGA configured via the FLASH memory. The only outstanding test is the I2C interface to the TTCrx, which will be tested in the next week or two. The schematics have been updated and checked against all problem reports and are in the process of PCB layout in the drawing office.

TCM: Six modules were manufactured; three are in use in the UK, one in Heidelberg. Five of these modules require a firmware update to correct a fault in the VME display. The firmware and the instructions are on the web site at

VME Mount Module (VMM): Six modules were manufactured. Three are in use, the other three modules need the existing connectors taken off and replaced with the correct VME connectors before they can be used.

TTCrx: The first batch (three) of non-rad-hard chips work o.k. and are now in use. A second batch (seven) of non-rad-hard chips was tested beginning of January 2003, and they do not work! These tests are on hold until the new batch of rad-hard versions is tested. Two cards were tested with the new rad-hard TTCrx chips. All functions are working correctly except for one. The sub-address lines are not connected from the connector to the TTCrx chip; these lines are used to latch the TTCrx ID on Reset. This function is not used (PROM used) on RODs or the DSS, therefore six boards that are assembled can be used on the RODs or DSS and will be tested next week with the new PROMs. The sub-address problem was corrected and a batch of 30 PCBs were ordered.

Test Modules:

DSS Modules: Ten modules are available to fulfil the test requirements. Four are in use at: RAL (2), Birmingham (1) and Mainz (1). Six need firmware updates and re-checking before use

GIO Card version 2: Design completed, out for manufacture end of this week. The GIO card consists of two parts: the back-end module with the Xilinx XC2V250-4FG256C; and two different front-end modules, one with SCSI connector for CMM and CP tests, and one with ECL to interface with the TTCVi (five single-ended and seven differential ECL)

G-Link Receiver Cards: A couple of modifications to the existing cards are required: change the SMA connectors to Lemo 00 connectors and include link-locked indicators. The schematics are not updated yet due to CAD version problems. Cadence are looking into this issue.


G-link Rear Transition Module for Heidelberg: Received specification from Paul via Tony in early January. The schematic is completed and it will be sent to Heidelberg for checking and approval before PCB layout commences.

Adapter Link Card (ALC:) for the TCM: A different adapter card from the CP/JEP is required for the Pre-processor and final ROD crate. No work has started on this yet.

Cluster Processor Module hardware - Richard Staley (slides)

The second CPM has been assembled , powered-up, and supply voltages checked o.k. They are now awaiting JTAG testing. The new module has a serious bow, with the centre out of line by 3mm.

On the first CPM, control and clock functions are working, but occasionally the FPGA Configuration Controller (FCC) Locks-up. [Afterwards , it was discovered that this happens after a configured CPM is reset. Following a firmware modification in both the FCC and Serialiser, this problem has gone away.]

The DAQ G-link output appears to be running error free, although the received frames contain an extra unwanted word. The CPM or DSS could be at fault.

Sixteen LVDS inputs are working without problems into the Serialisers. Uli mentioned that Mainz were having problems using LVDS links when the TTCrx was connected by fibre, but this was later found to be caused by a mis-behaving TTCrx card and not the LVDS parts. All 80 LVDS inputs will need to be tested together, and real-time bit-error rates measured. For this, 10 new LVDS source cards will be made; we have five assembled but untested. [One of these was used later that day to drive a JEM, without problem]

On-board 160Mb/s transmission is working well, although routing problems within the CP FPGA prevent the calibration and timing circuit working reliably. Typical waveforms of the on-board and backplane 160Mb/s links were shown. Backplane 160Mb/s signals suffer from reflections and attenuation, which reduce the voltage margins at the receiving FPGA. However , this shouldn't be a problem on the next PCB as the FPGA will use receivers with narrower logic thresholds.

There are two different termination techniques on the module. To reduce the pin-count on the backplane and Serialiser, each incoming signal, from a neighbouring CPM, feeds up to three destination CP FPGAs. Series/source termination would need active fanout to provide 384 point-to-point signals from the 160 inputs. This would require an extra 24 packages.

Richard commented on the timing margins of the 160Mb/s links in the light of problems with the CP chips 4-phase calibration. A 2-phase calibration scheme had been offered, but this is a non-starter as there is a chance that both phase choices could be very close to the transitions in the incoming data. Using only one clock phase overcomes the routing problems in the CP FPGA (as each input register no longer receives its clock through a multiplexer), and could work for the on-board data which is timed to within 1ns, but the backplane inputs are delayed much later and need another phase (or two). (The 1-phase clock design has since been used for timing scans)

A reminder that both the RAL and Bham systems are suffering from interference caused by powering-up the 9U crate supplies. The data sheets claim the combined surge is limited but could be 70A. At Birmingham, the adjacent CPU crate power-unit sometimes reacts to this by reseting the CPU. Fitting an inline mains filter has reduced the frequency of interference. Staggering the power-up of the two power supplies will help reduce this further.

CPMs #3 & #4 will be assembled soon. These may include faster variants of the CP FPGA.

To summarise: no reasons not to use the current module in the slice tests.

Cluster Processor Module tests - Gilles Mahout (slides)

The readot controller (ROC) of the CPM has been tested by sending RoI/DAQ data to a DSS populated with a G-link sink card. Data were recovered correctly but strange behaviour was observed: the length of data was always longer by an extra word, a copy of the last word. Bursts of data are recovered but not all of them, and the DSS memory does not seem to fill completely each time.

Concerning the CP chip and its links with the Serialiser, new firmware was released by James, where two phases are used instead of four. The number of errors has been reduced but there are still a few. A new version has now been written with just one phase and, providing the right delay has been selected via TTC between the Serialiser and the CP chips, all data are recovered correctly. A scan in time, by moving the previous clock between CP and Serialiser in steps of 104 ps, has been performed to identify the range where the CP chip receives correct data. The length of this time window is 3.5 ns, and the window repeats every 6 ns, which agrees with a data rate of 160 MHz. Similar patterns have been observed on two other chips.

Problems have been found on the Serialiser: the playback memory generates FFFF data from time to time. It does seem to happen to only one chip, and could be a defective chip or faulty connection. Unfortunately, during testing something happened which damaged the testing procedure. No error free zone has been recovered. This loss of regime has to be investigated.

On the software side, all previous tests have been performed thanks to the integration of the CPM services with the Online Run Controller software. It's foreseen to do similar tests with the ROC. In the long term, soak tests will be performed on LVDS Rx, with modified firmware in the Serialiser to do bit-error rate measurements.

Three new CPMs are to be available soon, one fully populated, and twoothers only populated with Serialiser chips. Full investigation of the backplane will then be performed, and the CP algorithm completely tested. More material is also needed to perform these tests, and a list can be found on the last slide.

In discussion, it was strongly suggested that a similar list of test equipment for use at RAL would help in our planning. The biggest discrepancy between what we need and what we have seems to be DSS modules and daughtercards.

Schedule - Eric Eisenhandler

Eric pointed out that in schedules discussed at the previous meeting, not enough notice was taken of the calorimeter installation timetable. We need to have at least a few Preprocessor Modules installed soon after the Tile barrel and electronics are commissioned, and so we cannot afford the luxury of assembling the entire Preprocessor before sending it to CERN.

Online software

JEM software and simulation - Thomas Trefzger

Thomas gave a brief summary of the software situation for the JEM at the tests going on at RAL. Scripts from Jürgen Thomas and Cano Ay were being used, and HDMC and module services software was being developed. A test-vector reader was being developed. The emphasis at Mainz has been on energy summation, but jet tests must also be added. As yet, no timing calibration software has been written, so UK CPM will be adapted.

Test vectors to simulate the realtime data path exist, together with expected results. The simulation needs to be integrated into Steve Hillier's framework.

Software summary - Murrough Landon

Murrough gave a brief summary of the recent work done by the various members of the software group before and during the Mainz/Stockholm visit to RAL. The main points were that the CPM tests are now running via the run control, TTCrx chips can be configured via the TTCvi, work was in hand to support DSS with two LVDS cards, and there was good progress on modules services and test vector readers for the JEM. Some tests were performed on RODs with new firmware, but so far these were not successful.

The most urgent tasks relate to multistep runs, e.g. to make timing calibrations in both JEM and CPM. This requires support in database, run control, and test vector readers. Work on CMM software should restart now that the first CMM is about to be released to us.

Murrough added that he is now a member of two TDAQ working groups. Documents and TDR chapters were being produced on the subjects of both monitoring, and error handling & fault tolerance. We should look again at these issues for our system – but time was too limited at this meeting. He was also invited to give a talk about our database requirements at the forthcoming database workshop.

Recent meeting highlights

Trigger/DAQ Steering Group - Eric Eisenhandler (slides)

Eric said a few words about the recent meeting of the Steering Group. He had given a long and thorough status report on the calorimeter trigger, including timetable and effort problems, and it had been sympathetically received. He also mentioned the line-up of meetings for the February ATLAS week, the forthcoming database workshop, and the deadline in May for papers at the IEEE annual meeting.

Any other business

Uli urged people to register for the Mainz meeting, as the hotel block booking runs out soon.

The next UK trigger meeting was tentatively set for Thursday, 27th February. Although planned for Birmingham, this was later altered to RAL.

Eric Eisenhandler, 3 February 2003

This page was last updated on 16 December, 2013
Comments and requests for clarification may be sent to