ClusterAlg.gif (577 bytes) The ATLAS Level-1 Calorimeter Trigger

[Home] [Architecture] [Meetings] [Participants] [Publications] [Software] [Search]

ATLAS-UK Level-1 Calorimeter Trigger Meeting

Thursday 19th April 2001 - RAL

Present: Panagiotis Apostologlou, Ian Brawn, Eric Eisenhandler (chair), John Garvey, Norman Gee, Tony Gillman, Bob Hatley, Stephen Hillier (minutes), Murrough Landon, Gilles Mahout, David Mills, Ed Moyse, Viraj Perera, Richard Staley.


Click for                                               Click for
summary                                            transparencies
                                                 *=ps #=pdf %=gif

Minutes of previous meeting

Hardware status summaries
CP "chip"...............................................Ian 10'
Generic Test Module...................................Viraj 10'
Cluster Processor Module............................Richard 15' #
CP/JEP ROD prototype..................................Viraj 10'
Common Merger Module specification...................Norman 10' #
Common Merger Module design.............................Ian 10'
Timing Control Module and adapters......................Bob 10' #
Crates and power supplies for slice test................Bob 10'
Summary PPM, PPr-ROD and MCM reviews...................Tony 15' #
Short-term schedule and reviews update.................Tony 10'

Testing and Integration
Work on Fujitsu CANbus solution........................Dave 10' #
Integration of CP/JEP ROD with RoIB and ROS..........Norman 15' #

Software status
Online s/w status & comments on T/DAQ week.........Murrough 15' #
Work on new trigger simulation software..................Ed 10' #

Other items
TileCal trigger-tower situation........................Eric  5' #
How many G-Links should we order? ==> ROD numbers
       and modularity revisited......................Norman 10' #
LEB and IEEE submissions...............................Eric  5'

Any other business
Dates of next meetings

Hardware status summaries

CP "chip" - Ian

James and Ian had built the VHDL test-bench for the CP chip code. James had recently succeeded in getting the test-bench running after sorting out a synchronisation logic bug, so he should get some results soon. It was clarified that all these tests were pure software simulation.

Generic Test Module - Viraj

Viraj summarized the situation as that there now was no GTM. Of the two modules sent for re-working, one was usable with some bits missing, and this had been given to the CMS group. The other was still unusable after a second re-working. There had been no feedback from the company on how the re-work had gone, or if any X-ray assessment of the failure modes had taken place.

Viraj now feels that the current boards will never work, but at least some lessons could be learned from the experience - in particular that re-working even a simple board may prove difficult. A new one will be built, and we will have to pay for parts up front and hope to claim back some of the costs. Some study of the pads on the failed boards could prove useful but perhaps the biggest concern is whether to stick with the current assembling company or try a new one. One of the reasons for initially choosing this one was that many others do not have the facilities for re-working modules. For the CPM we will need to build a good relationship with a reliable company, and to do this may require the production of a trial board first.

Cluster Processor Module - Richard

The current status of the schematics and PLD coding for the CPM were presented. The schematics are about 90% complete with the major incomplete areas being the Read-Out Controller, Hit logic and backplane connectors. Most of the coding was complete but some still had to be assigned to a device.

Richard said that the timescales were still vague, but he still hopes to get a module for testing at Birmingham by September. There would be a schematic review in early May, meaning it should be complete by mid-May. It was commented that allowing a reasonable time for testing, it seemed unlikely that the CPM would go to Heidelberg before 2002.

CP/JEP ROD prototype - Viraj

A decision needs to be made on assembling the two remaining PCBs and ordering four more PCBs for the slice test. The current problems as found at the integration tests should be reviewed before going ahead, but it was felt that there would be no major obstacle. The main concern was that all the problems should be identified as firmware bugs, and that no changes to the motherboard are needed. The two PCBs could be ready in two weeks, and the others would probably take about two months to produce.

Tony was interested in how long the ROD testing had taken as an indication of future work. About three full months were needed and so far only a couple of the FPGA programs had been tested, although the others should be easier now, but on the other hand, the work was not yet complete. Murrough asked about numbers of DSS modules - Viraj said that 9 existed, 3 having S-Link cards compared to 8 needed for tests. The modules are essentially ready to go, but more firmware may need to be written.

Common Merger Module specification - Norman

Norman presented the recent updates to the CMM specification. There were four areas that needed finalizing:
  • The Jet ET hits read out an RoI - Norman presented the new proposed data format including this information.
  • The backplane pinout - this was brought into line with Sam's backplane specification 0.9b2 which provides 3x27 available pairs for the 3x25 needed for the CMM system. There is a slight oddity with one pair which is split over two connectors, and Norman has suggested to Sam that he try to change this if possible.
  • The backplane/cable adapter board - a simple layout was shown.
  • The TTC registers - these are proposed to be the same as the CPM.
The new version of the specification includes all this work, and it just requires Sam's confirmation of the backplane change before it can be signed off. Eric commented that we currently have a lot of specifications that are not quite signed off.

Common Merger Module design - Ian

The original idea of fitting all the logic into one FPGA could not be implemented, so now the logic is spread over two FPGAs, and consequently these must be devices with large pin-counts. This meant that Panagiotis had to spend a lot of time entering the pinout, and Ian has been dividing the firmware into the devices. There should be spare capacity now with the split firmware. The work was mostly done, and a review was planned for the end of April.

It was suggested that the review should perhaps be a phone conference to include input from the Jet/Energy designers. Suggested reviewers for the schematics were Uli and Sam from the Jet/Energy side and Richard from the CPM side. There was a slot booked at the drawing office imminently, and Ian hoped that the design could be ready in time to not miss this slot. It was commented that nowhere in any specifications were the exact meanings of 25 signals defined. In principle this does not matter as they are FPGA to FPGA connections, but it was decided that they should be labelled bits 0-24 with 24 being the parity bit.

Timing Control Module and adapters - Bob

The TCM was now complete up to the stage of being fully routed, and the order was to be sent within a week after some final checks. Components were being ordered - five of each in the case of cheap components, and four for expensive in order to build four final working modules. The layout for the adapter cards was complete, and the routing was not expected to take long. There had been nothing new from Paul on the pre-processor version.

The schematics for the processor mounting module were almost complete and should go to the drawing office soon. Bob showed a picture of the mounting mechanics. The mechanical design was still in progress. Norman asked for confirmation that the adapter won't clash with a J0 connector.

News on TTCrx ASICs was that seven prototype ASICs had been found which could be used with the original TTC decoder card design. This means that 10 decoder cards could be made which should be enough to keep our systems running until the new design becomes available. Philippe warned that they needed to be heated before use. The new decoder card design itself still had to be done, but should be easy - it just requires a new device in the library and re-layout.

Crates and power supplies for slice test - Bob

There was no news from Sam on crates. On the power supplies, fives sets of part had been ordered with a three week delivery time. Four were to be made up and one set was for spares. One would be built at RAL, and three in Birmingham.

Summary PPM, PPr-ROD and MCM reviews - Tony

The PDR for the PPM and PPr-ROD had taken place on 8th March in Heidelberg. PPM specification draft 0.4 was reviewed for this module-0 design, and this included the first draft of the line-receiver daughter board (AnIn). The PPr-ROD is a module -1 design, and draft 0.1 of the specifications was reviewed.

The questions of the reviewers and others had been answered in comprehensive and well organised written responses, which helped the review to proceed quickly and amicably. Despite this, Tony felt in retrospect that although the PPM was covered well, there really was not enough time to fully cover the PPr-ROD in the same day. The general conclusions were that the PPM is now a well-specified design, although a lot of the functionality itself has been reviewed separately. The PPr-ROD and some of the components of the overall system (eg the Pipeline Bus) need more description and possibly separate reviews.

Tony listed more detailed conclusions for both the PPM and PPr-ROD modules which can be read in the transparencies. On the PPM, the most crucial of these was the grouping of BC-Muxed pairs incorrectly in eta, but this could be fixed by re-routing signals. Uli had some issues with the FCAL signal fanout, but this has since been resolved between Uli and Paul. On the PPr-ROD, the main area of contention was the Pipeline Bus and the complexity of the programming model. It was asked that more explanation of the design should be provided.

The other review that had taken place was the FDR for the Pre-processor Multi-Chip Module (PPrMCM). This was held on 12th April, using video and audio conferencing which worked reasonable well. The document reviewed was draft 0.1 written by Ullrich, whose imminent departure prompted the need for the FDR now. Due to problems finding a suitable reviewer from ATLAS Technical Coordination, Philippe was drafted in as the ATLAS panelist.

The document was an update on the PDR document from early 2000, and all of the PDR recommendations had been addressed. The test program was still evolving and good relations needed to be maintained with the production company, especially with Ullrich soon leaving. There was a concern about failure rates, with a worryingly high estimate given at one point, but real rates could not be known until the chip went into production. The details are again listed in the transparencies. Tony commented that the participation of Philippe was very important, and he could again be helpful for the PRRs.

Short-term schedule and reviews update - Tony

Nothing had changed significantly from previous reports except that we are running a little later again. It was becoming clearer that the UK modules would not be ready for a slice test before the end of the year.

Testing and Integration

Work on Fujitsu CANbus solution - Dave

Dave reported that he had recently made a breakthrough and could now generate and receive CAN frames with FADC voltages. He was continuing to work on tidying up the libraries, and trying to use remote frame requests. Future plans include investigating CANOPEN and SMBus, and trying to integrate all the features. Dave agreed that he would write an API description of his libraries, but right now most of the routines are empty.

Integration of CP/JEP ROD with RoIB and ROS - Norman

Norman firstly thanked Bruce for his enormous effort in making the tests possible, and also Viraj and James for their swift backup. Pictures of the integration test setup were presented, showing the main data paths - data from the DSS went via a G-link to the ROD, which processed and output the data onto two S-Links, which at various times were connected to the RoI builder, DSS and ODIN cards.

Norman summarized the results - many of the objectives were fully or partially achieved, with the main failure being problems at high speeds. However at low speeds, long runs with completely verified data were possible - these were actually better tests than had been achieved with the Muon system, in that different events were sent and received correctly, whereas the muon system only ever repeated the same event. Norman was keen to try a joint test with the muon system, but it appeared that their system was not ready at this time.

Many scope pictures were shown, all of which supported the correct operation of the system. Latencies were stable and in most cases predictable in terms of the amount of data transferred or simpler timing issues. One problem observed via the scope was an occasional G-Link glitch which needs investigation.

The main problem area came with S-Link flow control between the ROD and RoIB. Essentially there appeared to be synchronisation and buffering problems at both ends - new firmware improved the situation at the ROD end, but it is still not perfect. This problem meant that the system would hang at high speeds when the buffering system started to be needed. Careful choice of trigger rates, and bursts of triggers, was needed to avoid going into the flow control regime.

ROS tests were performed with the ODIN optical S-link. Continuous running up to 20kHz were achieved, with higher rates possible in short bursts. The muon system had managed to get to 40kHz, and it was not understood why we could not do the same. A joint test with ROS and RoIB reading data was attempted, and these ran for a over a million events before an error was found. Note that the ROS software used was just a simple test program and we still have no experience with the full ROS software.

Norman presented a long list of lessons learned from the integration test, which can be found in the transparencies. Probably another month's work was needed to solve most of the problems found. Norman concluded that the test setup at CERN was very good, and the people helpful. He felt that the experience was very helpful - it detected many errors that may otherwise not have been noticed. Further integration with the muon system and new RoIBs would be desirable.

Software Status

Online s/w status & comments on T/DAQ week - Murrough

News from the T/DAQ week was that new releases of software were continuing to be rolled out. Of specific interest, new event dump code should be available in June, and the TileCal presented their new monitoring tool. ROS software was now available for one S-link and should be extended to several S-links soon. There were also some organisational changes affecting ROD crate DAQ. There was interest in trying to better coordinate software across level-1, and a video-conference was suggested in May/June.

Murrough reported on the current status of online software. Bill has been ill, and Bruce had been mainly focussed on the ROD tests, so the main progress had been Murrough's proposals for changes to HDMC and integration of the HDMC hardware access into the DAQ system. A document exists proposing changes to HDMC in order to better meet our DAQ requirements, it can be found at:
Murrough had also redesigned the software website, now maintained under CVS in order that everyone can modify the site. This can be found at:
This contains much documentation of ideas for the slice test which people are encouraged to read and comment on.

On hardware, three more Concurrent CPUs had been ordered and had in fact been delivered to RAL. However, there is a newer version with better bus error handling which could be an advantage to us, although this version has no SCSI capability. Norman has been reading the documentation, and he and Bruce will make a decision over which CPU to buy. Norman commented that we need to understand the situation better as strange bus error behaviour was encountered at the ROD tests. Murrough was keen that we have a closer look at PC-ROS when we have obtained the necessary hardware. On this subject Norman pointed out how the DAQ software was still in a state of flux as we have never fully decided how it should be done - most of the ROD work at the integration test was unfortunately done with temporary ad-hoc software.

Work on new trigger simulation software - Ed

Ed reported that the Em/Tau code was now written, with thresholds being set via global values read from a text file. The code has successfully generated RoI data, and will soon be tested on Alan's test vectors. After that, he plans to start on the Jet/Et trigger code.

Other Items

TileCal trigger-tower situation - Eric

The tower summation problem may have returned, after apparently being resolved at the ATLAS week in February. Rupert Leitner was still worried about the effect on the Tau trigger. More discussion occurred at the T/DAQ week, and it now appears that the proposal still needs approval at a TileCal week in May. It is by no means guaranteed that we will get the option that we want, although one of the solutions may be to provide both our preferred summing scheme, and the capability to switch the other scheme. The LAr receiver stations had been reviewed on April 3rd, but Eric had not yet heard any news of it.

How many G-Links should we order? ==> ROD numbers and modularity revisited - Norman

Thoughts of ordering G-links for the whole system before/in case they go out of production had given rise to a re-think of ROD topology. G-link transmitter chip numbers are simply fixed by the CPM, CMM and JEM modules, but receiver numbers are not so obvious.

The currently documented and priced scheme requires 20 RODs with 16 G-links per ROD. However many of these G-links are only needed for ROD exchangeability, and not actually used. Norman has taken a more careful look at data rates for this proposal - details can be found in the transparencies. There were some inconsistencies in the original plans, for example CMM readout had been forgotten. Note that the cost of G-links is about 800 per ROD.

Depending on assumptions of how much data will be recorded (for example number of slices), Norman calculated that the maximum number of G-links needed would in fact be 17 for the Jet/Energy RoI ROD, and 18 RODs would be needed. Only 190 out of the total of 306 G-links would be required and powered. This configuration could just fit into one VME crate, but would probably be very uncomfortable, both on the crate and the front panel (two ODINs are needed per ROD for output).

Norman presented an alternative scheme with denser RODs using four ODIN outputs and each servicing a whole CP/JE crate DAQ output, rather than half a crate in the original proposal. These would require 18 G-link receivers each for the Jet/Energy crate. This would result in far fewer G-links - 216 of which 190 are powered. The reduction to 12 RODs would mean a far easier fit into one crate if they could still be single width modules. The main problem is that the front panel would be even more crowded, with no room for all the ODIN cards so they would have to be mounted directly onto the ROD, and signals brought to the front panel. Another issue here is that some of the modules would require the slower ODINs, so therefore plug-ability of different types is required.

Decisions will have to be made soon to determine the G-link order, although it was suggested we could buy enough for the dense option now, and more later if needed. Agreement with level-2 about ODIN strategy and with Sam and Uli must be sought (possibly by phone conference). Studies of engineering feasibility should also be done before any commitment to one solution. Norman will present these ideas again at Mainz, with some proposals for a cost sharing scheme.

A general discussion ensued about the relative merits of having one or two ROD crates. It was felt that the extra expense of a second crate and CPU was not significant compared to the difficulty of having a very full crate, especially as there is already other ROD information coming from eight other (PPM) crates, and more crates would be needing for test setups. The denser RODs do give a large saving on G-link expense, but this is also probably not very significant. Perhaps the 'modularity' of the RODs was a better criterion for a decision. It was felt that it would be nice not to have to worry about the two S-link speed options.

LEB and IEEE submissions - Eric

The IEEE deadline was imminent, and LEB the end of April. It was thought that there was nothing good enough for the IEEE at present, but ideas for LEB papers were one on the CMM design to illustrate FPGA versatility, and one on the ROD status. Abstracts should be circulated soon to be checked before submission. The muon RoIB tests had been written up as a paper, and it was suggested Norman should do the same for the ROD integration tests - the author list would have to be thought about.

Any other Business

There were no other issues raised.

Next meetings

Tuesday  5th June .............. UK meeting at RAL

 28th - 30th June .............. Joint meeting at Mainz

Tuesday 17th July .............. UK meeting at Birmingham

This page was last updated on 16 December, 2013
Comments and requests for clarification may be sent to