Funding

National eResearch Collaboration Tools and Resources (Nectar), Australian National Data Service (ANDS) and project partners.

Partners and collaborators

• Microscopy Australia

• Australian Nuclear Science and Technology Organisation (ANSTO)

• MASSIVE

• Monash University

• National Imaging Facility (NIF)

• University of Sydney

• University of Queensland

• University of Western Australia

• University of Wollongong

Description

The aim of this project is to make a significant contribution toward addressing the three major challenges identified in A Collaborative Australian Characterisation Informatics Strategy* (PDF, 120KB).

• Scale and complexity: Increasing data volumes and processing requirements; custom and specialised instrumentation requiring custom workflows; increasing need to perform analysis “in-experiment”; analysis of data from multiple instruments requires specialised skills and familiarity with the data.

• Working with digital objects: Data remains unpublished, is difficult to reuse, and it is often unclear whether it can be trusted; data curation is a second priority to publication and data is often non reusable; research software is often closed source and impossible to validate, can be challenging to newcomers and is often very specific to particular problems; research outcomes are difficult to validate and are often not reproducible; to increase return on investment research outputs need to be machine-readable.

• Expertise is rare: The characterisation community is increasingly reliant on data-science skills; digital expertise coupled with applied characterisation knowledge is rare; cross-modality analysis requires multiple areas of expertise; where knowledge is available it is often inaccessible beyond a local node or institution

The program is implemented through a number of activities, including:

• Making characterisation digital objects more Findable, Accessible, Interoperable, and Reusable (FAIR).

• Extending rich online environments for characterisation in the cloud and on HPC platforms for specific areas (e.g. creating or extending domain specific CVL Workbenches, building toward a national fabric for CryoEM processing and data management, creating a prototype CVL Windows desktop).

• Extending data processing workflows (developed in prototype under an earlier CVL project) to cover a broader range of instruments.

• Importing existing data processing workflows into CVL from the Australian Synchrotron.

• Augmenting an existing data management environment to publish and provide DOIs for data and connect data seamlessly to the CVL for processing.

• Working towards establishing Trusted Data Repositories

*The document A Collaborative Australian Characterisation Informatics Strategy (PDF, 120KB) was produced by a writing group with representatives from Monash University, Microscopy Australia, ANSTO and NIF, based on the outcomes of a series of open Characterisation Informatics workshops held in February and May 2017, involving stakeholders from Microscopy Australia, ANDS, ANSTO, BPA, Monash, NeCTAR, NIF, RDS, UMelbourne, UNSW, UQ, UoW, UWA, USydney, Agilent and NVIDIA.

Funding

Nectar, ANDS and project partners

Partners and collaborators

• MASSIVE

• Monash Micro Imaging

• Monash University

• EMBL Australia Node in Single Molecule Science - University of New South Wales

• Centre for Microscopy and Microanalysis, AMMRF-node - University of Queensland

• Research Computing Centre - University of Queensland

• ANSTO

• ARC Centre of Excellence in Advanced Molecular Imaging (2013-2020)

Description

The Characterisation Virtual Laboratory is part of a coordinated program to underpin imaging users and imaging facilities. This program is implemented through a number of activities:

• Integrating Australian instruments with national-scale data management infrastructure and projects.

• Developing rich environments in the cloud and on HPC platforms.

• Improving software so that researcher access is simple and seamless.

• Developing instrument data workflows for accelerated discovery and trusted data.

The goal of this effort is to allow instrument facilities the ability to tell their users:

“Your data is now being ingested by a nationally connected data management system and from there you can easily access it in an online collaborative environment that has many of the tools and services that you need to gain insight.”

This coordinated proposal to Nectar and ANDS is to:

• Integrate BILBY, a small-angle neutron scattering instrument with Store.ANSTO (ANSTO’s MyTardis deployment) and develop a prototype Neutron Scattering workbench in the CVL for the analysis of data from this instrument.

• Continue operations of the Characterisation Virtual Environment on the Nectar research cloud, the outputs of which has now been used by over 600 Australian researchers.

• Investigate how to federate the CVL across multiple sites and service providers.

• Integrate Lattice Light Sheet Microscopes (LLSM) and improve the trustworthiness of LLSM data.

Funding
Nectar and project partners.

Partners and collaborators

• Monash University

• MASSIVE partners: Monash University, CSIRO, Australian Synchrotron

• ARC Centre of Excellence in Advanced Molecular Imaging and ARC Centre of Excellence for Integrative Brain Function

• CSIRO

• Centre for Microscopy, Characterisation and Analysis - University of Western Australia

• National Imaging Facility - University of Queensland

• Melbourne Centre for Nanofabrication, ANFF-node

• Australian Microscopy and Microanalysis Research Facility - University of Sydney

Description

Remote desktops have proven to be a successful method to help researchers access tools and data on the cloud and alongside HPC systems. In particular, they have proved highly effective for:

• Supporting communities that are relatively new to centralised computing infrastructure (for example, the neuroimaging community as demonstrated as part of the CVL and MASSIVE)

• Enabling researchers who are working with increasingly large data sets that cannot be moved to the local desktop for analysis and visualisation (for example, researchers working with high resolution volumetric data produced at the IMBL at Australian Synchrotron)

• Provisioning a wide range of existing desktop applications centrally, without the need to wrap tools in a HTML interface to expose them on the Internet.

Project details:

• Improved desktops services and tools so that they are easier to deploy and so that they can be deployed to a broader range of Australian research communities, including two new computing sites: The Brain and Mind Research Institute, at the University of Sydney and IVEC / University of Western Australia

• Undertook work to extend the Scientific Desktops in the Cloud model to the Terrestrial Ecosystem Research Network

• Supported a wide range of existing users and existing deployments of CVL technology

• Deploy CVL in Western Australia on local infrastructure to support the Centre for Microscopy, Characterisation and Analysis (CMCA) at the University of Western Australia (UWA) and the broader Western Australian imaging community. The CMCA is both a node of the National Imaging Facility (NIF) and of the Australian Microscopy and Microanalysis Research Facility (AMMRF)

• Deploy CVL technology to CSIRO Scientific Computing

• Improve CVL to make managing multiple instances easier and more scalable

• Develop a plan for a CVL-like environment for the Australian National Fabrication Facility Design House

• Continue operations of the Characterisation Virtual Environment on the Nectar research cloud.

Under this project, the MASSIVE Desktop and CVL user base grew to 590 unique researchers, plus we continued new deployments at new sites (not included in this count).

Researchers combined have run over 23,000 desktop sessions on CVL on the Nectar cloud, M1 and M2 systems on MASSIVE and the new Monash MonARCH system. This number is likely much greater if taking into account CVL software being deployed at other sites.

Funding
Nectar and project partners

Partners and collaborators

• The Brain and Mind Research Institute - University of Sydney

• IVEC and University of Western Australia

• Terrestrial Ecosystem Research Network (TERN)

• University of Queensland

Description

This project builds on the Nectar Characterisation Virtual Laboratory (CVL), and in-particular two parts of the project, Instrument Integration, and the CVL Scientific Desktop environment:

Instrument Integration:

Over 20 microscopes are now integrated with the CVL, meaning that imaging data produced on the instrument is automatically captured, managed and delivered to the cloud for processing, analysis and visualisation. However, deploying the CVL to more instruments has proved demanding: it is now apparent that one of the major bottlenecks delaying the integration of a larger set of instruments is that setting up control PCs for data capture is manual and challenging - in particular for facility managers. This project will develop an instrument integration app, called MyData, that will make integration quicker, simpler and less reliant on specialist IT-support.

As a result of this project, MyData is now used at 35 instruments, across 10 facilities, at 4 institutions, in addition 17 further instruments are planned at 4 institutions.

Number of instruments integrated via the MyData application:

• Production: 32

• Pre-production: 3

• In progress: 5

• Planned: 17

Types of instruments integrated:

• Microscopy: Advanced light microscope, Fluorescence and Confocal microscope, Multiphoton microscope, Cryo-electron microscope, Scanning electron microscope, Transmission electron microscope, Atom Probe

• Flow Cytometry: Cell Sorters, Analysers

• Imaging: X-ray microscope

Remote Desktop:

Over 500 researchers are now using the remote desktop environment and associated tools created as part of the CVL. The CVL Remote Desktop provides a tool-rich environment to process, analyse and visualise imaging data. This project will undertake key improvements to the CVL to increase usability for both new users attained through the instrument integration program and existing users.

Strudel-Web is now deployed across 2 sites (TERN and MASSIVE) and is being used to access resources from MASSIVE, CVL, MonARCH, and TERN.

This project is integrated with the ANDS ImageTrove project which is deploying MyTardis across National Imaging Facility sites, and the RDSI ReDS3 Characterisation allocation.

The effect of this project will be a significant increase in the instruments that feed data to national eResearch infrastructure, and a new community of researchers and facilities that will use infrastructure developed under the CVL.

Funding
Nectar and project partners

Partners and collaborators

• AMMRF (now Microscopy Australia)

• ANSTO (Australian Synchrotron)

• Australian National University

• MASSIVE

• Monash University

• NCI

• NIF

• The University of Melbourne

• University of Queensland

• VPAC

Description

The '21st century microscope' will not be a single instrument; rather it will be an orchestration of specialised imaging technologies, data storage facilities, and specialised data processing engines. Currently, these components are distributed across Australia with little software infrastructure to help scientists navigate between them.

The CVL Desktop will provide a managed scientific desktop – an interactive eResearch environment for analysis and visualisation of multi-modal and multi-scale data. The CVL will gather and integrate existing tools and techniques and will run across the NeCTAR Cloud, specialised systems such as MASSIVE, as well as other computing and data storage facilities. Integrating the desktop with these facilities will provide the research community with easier access to high-performance computing and large data collections. The main deliverables from the CVL project will be the CVL fabric for characterisation facilities and the capability to deploy this fabric to a broad range of research areas.

The Characterisation Virtual Laboratory (CVL) (Monash University) proposal brings together a number of tools for data capture and management, and data analysis and visualisation.

Data capture and management tools will feed data and metadata to the CVL. These tools include:

• Tardis (Monash University)

• eResearch Tools for the Australian Synchrotron – VBL and Mecat (Australian Synchrotron)

Analysis and visualisation tools for image and volume data will enrich the CVL environment:

• Cloud-Based Image Analysis and Processing Toolbox (CSIRO)

• Drishti and Voluminous – Volume Rendering Tools (ANU)

The tools and VL discussed here are complementary and will create a powerful environment for the characterisation community.