ATLAS-UK Level-1 Calorimeter Trigger Meeting
Thursday 17 April 2003 at RAL
Present: Bruce Barnett, Ian
Brawn, James Edwards, Eric Eisenhandler (chair), John Garvey, Norman Gee, Tony
Hillier, Gilles Mahout, Tamsin
Moye, Viraj Perera, Weiming
Qian, Richard Staley, Jürgen
Thomas, Alan Watson
Click this side Click this side
for summaries for slides (pdf)
CMM hardware and firmware status........................Ian
ROD hardware and firmware status......................James
ROD test status.......................................Bruce
Bit and pieces........................................Viraj
CPM hardware and plans..............................Richard
New TTC decoder, VME tests and VMM..................Weiming
UK tests: Integrating CPM, CMM, ROD.......Norman/Discussion
Recent meeting highlights
Discussion: Working group organisation & tasks.........Eric
Front-end rack, patch panel and cable layout
9U ROD design
Organisation of the project
Any other business
Date of next UK meeting
CMM hardware and firmware status
- Ian Brawn
Ian reported on the status of CMM firmware and hardware. He
developing the Jet Energy Summing firmware. Andrea Dahlhoff has supplied
algorithm block for the Jet Energy Summing firmware and Ian has added the
parity, synchronisation, DAQ and RoI logic, modified from the CP Counting
firmware. Currently the design is being simulated and bugs fixed. Ian raised
the question of medium and long-term responsibility for this design, as
has now left Mainz; Eric will ask Mainz about this.
On the hardware side, the new
CMM has returned from assembly. Adam Davis will fit heatsinks, etc, and the
board will be JTAG tested next week. Main areas of interest for the board
FPGA-programming logic, both of which are new. Real-time logic and Readout
tests will proceed as for the first CMM. A Trigger Processor crate will
required in the electronics lab for all of these tests.
ROD hardware and firmware status
- James Edwards
James had little to report on the ROD firmware, as
important work was being carried out on the future CP chip design. James
reminded the meeting that the outstanding five specifications for future
variants of ROD 0.1 (and ROD 0.2), i.e. the 6U modules, were required before
further progress with new firmware versions could be made.
The six new RODs are still waiting for G-link receiver cards;
see Viraj's talk below.
Norman suggested that he, Bruce and Steve should look at existing
firmware specifications to make sure they are still ok, and notify James
of any changes.
ROD test status - Bruce
Bruce presented an update on aspects of
ROD testing. He noted that
three problem reports (pertaining to the new fiwmare variants) were awaiting
He is re-evaluating the performance of the CP-RoI firmware, but regards the
firmware as stable. New ROD controller firmware (as opposed to Data Firmware,
contains the source-module specific parts) has been made available, but has
effect in resolving the problem reports.
Bruce again has two test stands set up, but finds hardware
problems makes it difficult
to maintain them both. Intercoupling of software, firmware and hardware problems
is at the heart
of the issue.
He noted that in order to proceed to tests with real source
modules (JEM, CPM),
some outstanding minor issues should be resolved. It is intended to collate
these in conjunction with Steve and Norman. A particular issue is the
Module ID in the real JEM datastream.
He commented that there was an issue to be solved in that
the specification of the ROD made during implementation had not been
fed back into
the specification. In addition, some better bug-tracking mechanism
is desirable. It
was felt that notification by e-mail of firmware users of the existence
releases would be useful.
Bruce finished off with a speculation about the use of G-Link
Lemo-00 connectors planned for use on the ROD. He queried whether,
such trivial cases, re-certification efforts should not be made.
Bits and pieces - Viraj Perera (slides)
TCM: The pre-processor and the
final ROD crate require a different ALC (VME compatible), which hasn’t
been designed yet. A draft specification was released for discussion on the
17th of March 2003. Meanwhile a schematic
has been entered, but is waiting for the approval of the specification to
complete the job. A module is required by June 2003.
TTCrxDec: 35 modules
manufactured; some are in use, some require testing.
TTCrx decoder cards: New cards for the
prototype ROD modules have been designed and 10 PCBs manufactured (in red
be used only on
the RODs). 10
TTCrx chips have been ordered from CERN for these modules
DSS Modules: Eight new DSS modules
were ordered on the 14th of March after the solder resist problem was corrected.
are in Assembly
back at RAL end of April.
Out of the first batch of ten, two still require attention (problem with
The front panels of all the DSS modules will be modified to take the
TTCrx signal via a double-pole Lemo 00.
GIO card: Two modules (back-end) successfully tested with
the two frond-end modules (LVDS-SCSI and the ECL). The remaining six modules
are in assembly
(due end of April)
G-Link cards: For G-Link Rx (needed
for RODs) and Tx, did the necessary changes on paper and handed it to DO
to add the changes
to the existing layout (not
Layout underway now after upgraded-revision problems.
Glink Rear Transition module for Heidelberg: Schematic
completed, queued in DO (next after G-link cards)
CPM hardware and plans
- Richard Staley (slides)
Status: We now have three assembled CPMs. One fully
working, the second with unusable CP FPGAS due to assembly problems but otherwise
operational, used to drive data over backplane to the first CPM. A third
needs preparation before it can be JTAG/boundary-scan checked for any assembly
A fourth bare PCB has been delivered, which will be assembled without CP
chips. PCBs delivery was delayed due to low yield.
TTC inputs: CPM assumed parallel termination, but
TCM outputs were source terminated (as specified) so CPM termination removed.
termination resistors were found to have the wrong values for the
specified 100-ohm tracks across the backplane. These were 56 ohms, and
should be replaced by 39-ohm resistors. The output biasing resistors
also need changing to 390 ohms. If agreed by the collaboration, then an
Engineering Change Order will be issued to modify the design of all
Note that other modules (PPM, JEM etc.) should also check this!
9U Crate Power Supplies: Three PSU boxes have now
been modified to include a mains filter and delay circuit. At Birmingham,
9U crate no longer interferes with surrounding equipment, however Bruce
mentioned that at RAL one item of equipment is still affected by the
power-on of their supplies.
Next version of CPM: Because of timescales, we
previously decided to keep any changes to a minimum, so the production CPM
will continue to
use 8 VirtexE CP FPGAs. The main task will be to re-route all the CP
input signals for better quality. Clock distribution will be improved by
using more PLLs and attention to layout. The PCB will hold a new TTCdec
card with an improved, but different connector. There was a suggestion
to mount the TTCrx chip directly on the CPM (dropped following discussion at
a subsequent phone conference - EE).
Richard would like the CP FPGAs to use their unconnected Vref inputs so that
other signalling standards, such as SSTL2, may be used. A new CPM with
this option would still be 100% compatible with the previous version,
and even use the same firmware, allowing new versions to be tested in
existing crates. The benefits of SSTL2 are lower switching currents
giving less noise, and better-defined signal thresholds giving improved
CP FPGA clocks: We need one clock to capture the
onboard data and at least one more delayed clock to capture data from adjacent
just the skew due to layout and transmission path differences, the
difference between having one clock and having two clocks is small, 1.2ns
vs 0.9ns. There is no significant benefit in having three clocks, as the CP
chips see onboard data skewed by 0.7ns (due to the two columns of
Serialisers). I felt one clock would be adequate, but agreed to design
the PCB for two 'backplane data' clocks.
Timescales: RAL Drawing Office will be able to
start work on the re-design early July, and estimate layout finished mid-end
Assembled boards should appear before October.We must complete testing
of the present design before August. ROC and HIT outputs have not been
connected to real modules, and the CAN microcontroller remains untested.
The new modules must be debugged and tested by mid-2004 for the FDR.
tests - Gilles Mahout (slides)
Extensive tests have been done on the second CPM. Only four CP chips are connected
and so far they have not been tested. A TTC scan of the LVDS input data
coming from DSS has been done and shows similar results as CPM no. 1, with
all LVDS receivers locking OK. Data have been exchanged between both CPMs,
CPM successively plugged on right and left hand side of CPM no. 2. Data have
been correctly recovered and TTC scan performed without any problems on
input links. Some bad channels that were seen when using the loop-back boards
to have disappeared.
Three more DSSs have been delivered to Birmingham and a CPM could be 3/4
fully populated with LVDS inputs. Both boards are now populated with a new
TTCdec card with I2C access. An HDMC I2C part has been implemented and provides
virtual mapping of all TTCrx registers.
The setting of different clocks for the TTC scan could been done per
board now, without requiring a TTC broadcast.
Work has been done to synchronize data from playback memory between
the two boards. Similar work has to be done for the DSSs but unfortunately
one DSS has an old TTCdec card which makes it difficult to have all four
working together. Richard has modified the distribution of the Deskew1
clock on board for better jitter, and the operational time window is
now more than 1 ns for data coming from the backplane.
First BER tests have been done on one channel of one serialiser, with a BER
of 10–13. The pseudo random pattern is only 128 long and
the firmware should
be changed to a random pattern similar to the one used in the DSS. More
BER tests are foreseen, with crosstalk measurement and as a function of the
position in the crate. The next step will be to integrate the CPM with a ROD
a CMM at RAL. Although the Readout path has been tested earlier, long-term
tests need to be made.
Gilles remarked that the LVDS cables going into the backplane are not well-supported,
and as the number of cables in use increases we will need a good mechanical
structure to provide strain relief.
New TTC decoder, VME tests and
- Weiming Qian (slides)
TTCrx vs TTCrxDec: The definition
and classification of jitter was given. The relation of clock jitter and
BER was presented.
System jitter requirements and many details
discussed. An example of a jitter test from previous experience was given.
Serveral approaches to a better solution than the present
TTCdec were presented. We could have all chips on the main module, TTCrx
on a daughter-board but PLLs on the main module so that the clocks are
cleaned up right near where they are used, or everything on a daughter-board
with care taken to feed the clocks to the board cleanly via good connectors
and differential lines. This last option would produce a rather large daughter-board.
In discussion within the UK meeting the first or second options seemed
favoured, but we need to get the views of Heidelberg (Paul) and Mainz (Uli)
come to a decision. However, the timescale is short if the new boards are
to be used on the PPM and new CPM.
VME-- signal test: Several
VME-- signals were tested on the 9U backplane. The results show overshoot
30% and undershoot about 10%. Futher tests will be needed
when the crate is fully populated.
VMM status: VMM modules are fixed.
The problem is identified as the +5V fuse, which caused
a significant voltage drop of about 150mV. The solution for now is to bypass
the fuse with thick wire; for the future lower resistance fuses should
UK tests: Integrating
CPM, CMM, ROD -
Norman Gee/Discussion (slides)
Norman initiated a discussion on testing. There is now a
spectrum of testing, from single modules to growing subsystems (viewed both
as subsystems and as tests of a single module's interfaces), rather than the
planned sequential approach. However, before FDRs can be passed,
must pass tests with all nearest-neighbour module interactions working. As
far as the CP subsystem is concerned, it is urgent to start CPM–CMM and
CPM–ROD tests. These will be started at RAL as soon as possible. Thereafter,
work at Birmingham should focus on soak-testing with all real-time and
readout paths busy (including CMM), and at RAL on system aspects including
ROD output and CMM–CMM interactions. Both
institutes will need both modules.
- Steve Hillier
Recent work has concentrated on trying to properly integrate the
current module testing (particularly the CPM and JEM) into the
run control environment. This is already partially done, but needs
more work. A recent meeting and various discussions were held at
QMUL while Cano was visiting, and just after Jürgen had started
work there. Jürgen will be working on the JEM simulation when the
recently received code from Stockholm has been tidied up by Steve.
Recent meeting highlights
ROD-crate DAQ - Norman Gee
Norman gave a brief summary of the ROD-Crate DAQ meeting at CERN on 26 March
2003. For details see slides at http://agenda.cern.ch/age?a03444.
There were various announcements. CERN will support diskless booting of
TTC support has been re-organised after retirement of B. Taylor. M. Joos
is the contact for the VME modules. ROD Busy Module drivers and library
documentation is in EDMS. It was confirmed that
there is no ATLAS-standard ROD Crate TTC distribution scheme.
start CTPd integration tests from July 2003. He has a
Concurrent CPU. The CTPd requires a CERN 9U crate. He uses a feature of
ROD-Crate DAQ in which software pulls data over VME from the CTPd into a
Software ROB-in, then transmits it over an S-Link from the single-board
Transmission can also be over Ethernet. The system can sustain 20kHz of
80-byte data to S-Link.
There was also some discussion of calibration,
and input is
Level-1 on our requirements for the next DIG forum.
ROS - Bruce Barnett
Bruce presented a status report on issues involving the use
for L1Calo readout, an area in which he acts as contact point for the group.
He reported a meeting which took place on 27 March, just after the ROD-crate
DAQ meeting, which aimed at a clarification of issues of monitoring and
acquisition relating to L1Calo use of the ROS. The discussions were
useful, and led to the proposal to use a customized dataOut() in the IOManager
to help select events. A later discussion with Markus Joos and Jorgen Petersen
led to the suggestion to move to FILAR/HOLA technology for readout. This
would require some investment, but would keep us in line with mainstream
readout developments, and improve throughput in our system.
System reconfiguration is underway, and it is hoped to reinstall
ROS software quite soon.
TileCal receivers - Eric Eisenhandler (slides)
Eric gave a brief summary of recent interactions, including a discussion at
CERN, with Bill Cleland. A complete first draft of the receiver specification
and is also available on the web for everyone else to see and comment on, at http://hepwww.ph.qmul.ac.uk/~efe/ReceiverSpec.pdf.
The big unresolved issue is whether transformer coupling is acceptable for
the unipolar pulses of the TileCal, where they will introduce a small baseline
shift. Bill thinks it is probably ok, but to calculate the likely shift we
need more information on the rate and amplitude distribution of pulses. A request
for what is known has been sent to the TileCal group. Eric is also trying to
contact the Rio group, who built the summing amplifiers, to be absolutely sure
we understand the gain of the signals – their web pages are a bit ambiguous,
and it is easy to make factor-of-two mistakes when dealing with differential
The UK has ordered the variable-gain amplifier chips (which are going out
of production) needed for the receivers, after an unhappy experience with a
firm quoting an incorrect low price and then deliberately 'losing' the order.
The are also indications that the TileCal group is preparing to go out to tender
for the long cables from the calorimeters to USA15 immediately after Easter.
Discussion: Working group organisation
and tasks (slides)
The items below were identified at Mainz as tasks for the coming
few months. To make progress, working groups should be set up to examine
the issues and
make recommendations. Although not UK-only, they were discussed here to get
UK views and to start naming the people who would work on them. Note that
in all cases other volunteers would be more than welcome.
Front-end rack, patch panel and cable layout
This item was triggered partly by realisation that we had not thought about
specifying and ordering the short cables from TileCal patch panels to receivers,
smaller number of cables from receivers to the downstream patch panel (needed
to avoid "octopus" cables). In addition, there had been a discussion at CERN
on cable mechanics that was
relevant (notes circulated by Paul Hanke),
of course we also
need to minimise our latency.
The layout should provide ease of access to patch panels, receivers and Preprocessor,
and good cable mechanics for support and strain relief. It should then be possible
to specify the lengths of the short cables so that they can be ordered at about
the same time as the long TileCal cables. The latency must be kept to a minimum.
Hopefully a suggested layout could be produced within a month or two, and then
distributed for comments by the community before coming to a conclusion.
This exercise should also be used to trigger completion of the cabling document
produced by Steve and Murrough last year.
The suggested people for this were: Murrough, Steve, John, and Paul.
9U ROD design
We must define the specification for the 9U full-specification ROD, leading
to a Preliminary Design Review. It should
incorporate what we have learned so far, plus the requirements of Preprocessor
readout – including compression of FADC raw data.
We probably cannot decide yet on processing requirements for calibration
and monitoring, so a way around might be to make
provision for a processor or DSP daughter-board to be defined later.
The suggested timescale: is 2–3 months.
Again the suggested procedure is to
set up a small working party,
and invite all interested people to brainstorming session(s) before writing
a draft specification.
John questioned the need for data compression as he felt that we would not
often want to read out a large volume of raw data. Viraj and Ian suggested
starting from the specification for the present 6U ROD, replacing 6U with 9U
and then seeing what needed to be changed.
The suggested people for this were Viraj, Bruce, Norman and Ian from the UK.
Sam and Uli might be asked from outside the UK. Steve and Paul are currently
very busy but might be suitable as reviewers (along with others).
Recent measurements on the CPM and CMM have been worrying. We need to try
to reduce latency wherever possible, but not to waste precious effort where
there would be little gain. It was suggested that a group of people should
take an overall look at the problem. First to summarise all present measurements
of latency, and define how to estimate or measure missing information.
Then to consider possible areas for improvement, such as
rack and cable layout,
choice of FPGA device types, and
firmware. The evaluation should try to establish where we could make really significant
gains in the latency, in order to understand where the problems are and to
make best use of our effort. The timescale again might
Suggested people were Tony, Murrough, and Gilles from the UK. Uli and Sam
might be asked, and for the Preprocessor perhaps Pavel.
Organisation of the project
Although triggered by Tony's request to give up hardware coordination
duties, this has broadened since our organisation has not been reviewed since
it was set up about six years ago and the nature of the work is now quite
tasks need to be re-defined (also see below) and for duties needed short-term
we need to provide interim cover. For the contact people at our "outside interfaces"
we should evaluate their effectiveness and
of their roles (which contacts are now needed, etc.). For
coordinators, we should review the structure now needed for hardware, software,
A small committee of senior people should be set up by Eric, with the aim
of presenting proposals to the Management Committee at the Queen Mary meeting
in July. The entire community should be asked for frank comments by Eric. Eric
people and coordinators to see how they view their roles and effectiveness.
Contact people should be asked to summarise their activities.
We should also
make sure that when working groups outside the project would like a representative,
it is us and not them who decide on who it is.
Tentatively set for Monday, 19th May at RAL.
6 May 2003