Cloud technology has the potential for widening access to high-performance computational resources for e-science\nresearch, but barriers to engagement with the technology remain high for many scientists. Workflows help overcome\nbarriers by hiding details of underlying computational infrastructure and are portable between various platforms\nincluding cloud; they are also increasingly accepted within e-science research communities. Issues arising from the\nrange of workflow systems available and the complexity of workflow development have been addressed by focusing\non workflow interoperability, and providing customised support for different science communities. However, the\ndeployments of such environments can be challenging, even where user requirements are comparatively modest.\nRESWO (Reconfigurable Environment Service for Workflow Orchestration) is a virtual platform-as-a-service cloud\nmodel that allows leaner customised environments to be assembled and deployed within a cloud. Suitable distributed\ncomputation resources are not always easily affordable and can present a further barrier to engagement by scientists.\nDesktop grids that use the spare CPU cycles available within an organisation are an attractively inexpensive type of\ninfrastructure for many, and have been effectively virtualised as a cloud-based resource. However, hosts in this\nenvironment are volatile: leading to the tail problem, where some tasks become randomly delayed, affecting\noverall performance. To solve this problem, new algorithms have been developed to implement a cloudbursting\nscheduler in which durable cloud-based CPU resources may execute replicas of jobs that have become delayed. This\npaper describes experiences in the development of a RESWO instance in which a desktop grid is buttressed with\nCPU resources in the cloud to support the aspirations of bioscience researchers. A core component of the architecture,\nthe cloudbursting scheduler, implements an algorithm to perform late job detection, cloud resource management and\njob monitoring. The experimental results obtained demonstrate significant performance improvements and benefits\nillustrated by use cases in bioscience research.
Loading....