• Home
• Articles Archive
• Features List

• Media Pack
• Contact Us
• Subscribe

Brain Power

From STORAGE Magazine Vol 6, Issue 7 - September 2006

A state-of-the-art Apple Xserve cluster is providing the processing punch required by the York Neuroimaging Centre to conduct 'serious science'

The neurons in the human brain are connected by approximately one trillion - that's 100,000,000,000,000 - synaptic connections. The task of researching this activity is enormous.

This is what work at the York Neuroimaging Centre is all about. The unit's two high-end scanners generate up to 2GB of data per hour, and the technology working behind the scenes to process it is an Apple Xserve cluster with 52 dual-processor nodes.

The Centre opened in May 2005 and forms part of the Department of Psychology at York University. It uses non-invasive imaging methods such as MRI (magnetic resonance imaging) and MEG (magnetoencephalography) to study the structure, chemistry and function of the human body - in both a healthy and diseased state - and particularly of the human brain.

"We do lots of work on brain connectivity," explains Professor Gary Green, director of the Centre and professor in the Department of Psychology. "Imagine the brain is divided up into 50,000 boxes; we're trying to work out how each box talks to each of the other boxes. That's 50,000 boxes talking to 50,000 boxes, which is a lot of connectivity to analyse."

The Centre undertakes both clinical and commercial work (under the trading name YNiC Ltd), as well as conducting research in the evenings; in short, there's no such thing as down time. "The scanners are never turned off," says Professor Green. "It cost £5.2 million to set up the centre and £250,000 to run it per year - to justify that, we run the scanners day and night."

The computational element has clearly been key to the success of the entire venture. Green says: "We wanted a high-performance computing solution that could cope with many potential users and fast turnarounds. In the past, processing jobs could take weeks and yet much of the analysis is fine grained and very repetitive, so we were looking for a system that could handle large data sets in parallel."

He adds: "The solution also had to be gridable, scalable and evolvable - ideally off-the-shelf. And last, but by no means least, it had to be cost-effective."
Having decided upon the specification, an independent survey was conducted by a team of computer scientists. Windows, Linux, Mac OS X and Sun technology were all assessed according to a fixed set of criteria.

A long list of standard brain imaging software was tested on them, including Matlab, Atlas, Maple, Brainstorm, Neurolens, Brainvoyager and MPI. More general software programs were also tested such as those for firewalls, databases and email. Technology was also assessed in terms of security, back-up and support, future-proofing and upgrade paths, contingency planning, and cost.

"Apple won hands down," says Green. "It was the fastest and easily the cheapest option in terms of both storage and back-up and computing."

"Contingency worked well - if it turned out we didn't like Mac OS X, we could use Linux. It also ran every single one of the software packages we required. The support we received was good and there was a clear path in terms of what Apple was doing with software. The whole thing was far and away the best solution."
The Centre's overall investment in Apple technology amounts to £300,000 and it is currently the biggest G5 Xserve cluster anywhere in the UK. Among the Centre's hardware setup are 8 Xserve G5 servers, 10 Power Mac G5s, 34 iMac G5s, 2 5TB and 2 3TB Xserve RAIDs (mirrored in another building for 32TB of total storage), 2 Opteron Linux Firewalls, 2 HP 128-port Ethernet switches, a 32-port Fibre channel switch, a HP tape library and various printers, faxes, photocopiers and digital cameras.

Green is particularly pleased with the performance of the Xserve cluster. He says: "We've already adapted software to the cluster so that it can perform proper parallel computing. Jobs that previously took days now take minutes, even seconds. This obviously makes a huge difference in terms of workflow and productivity."
Integration was another key point. Green says: "We've been able to incorporate a Sun grid engine so that the Xserve cluster can support lots of jobs. Because Apple provided us with an off-the-shelf solution, we were able to have the Sun software up and running in four days." Similarly, integrating the HP jukebox was literally a matter of 'plug and go'."

In order to increase reliability and resilience, the Apple RAID array has now been augmented by a SAN storage system, mainly for archiving purposes. "We identified certain bottlenecks, and to solve them, we introduced the SAN." Because the imaging scanners run 24/7, the Apple kit's reliability is a big issue, but as Green says, "Since October last year, we've only had two broken fans on the iMacs and one RAID disc that went down. Most servers have been up and running for 280 consecutive days without any problems."

Green adds: "We've had to expand the setup significantly since the initial investment. The demand from users is increasing all the time." He cites one user who was generating 2GB of data every minute while undertaking research into brain connectivity.

"In the future we'll expand even further," comments Green. "We're looking to support a wider group of users including a Europe-wide database - we also want to introduce support for external users and to create links to external grids. Apple has been a good choice for us and won't compromise our future development."
Green says: "This is serious science. We've got people modelling quantum mechanics at atomic level right through to the processing of human brain data." And yet, despite the seriousness of the endeavour, he concludes: "Because of Apple, setting up and running the technology in the Centre has been fun rather than stressful."