{"id":2280,"date":"2014-10-21T11:26:25","date_gmt":"2014-10-21T09:26:25","guid":{"rendered":"http:\/\/news.embl.de\/?p=2280"},"modified":"2021-07-13T12:01:00","modified_gmt":"2021-07-13T10:01:00","slug":"1410_data-move","status":"publish","type":"post","link":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/","title":{"rendered":"Job dispatched"},"content":{"rendered":"\n<p>EMBL-EBI, home to the world\u2019s most comprehensive collection of life-science data, ran an unprecedented 11.9 million jobs for its users throughout the world in September 2014. At the same time, it began to move hundreds of its servers from two locations in London to a new data centre at Hemel Hempstead. So how does the institute continue to serve up petabytes of data while on the move? Very, very carefully, says <a title=\"Steven Newhouse, Head of Technical Services\" href=\"http:\/\/www.ebi.ac.uk\/about\/people\/steven-newhouse\" target=\"_blank\" rel=\"noopener noreferrer\">head of Technical Services Steven Newhouse<\/a>.<\/p>\n\n\n\n<p>\u201cThe move to Hemel Hempstead has to be very tightly choreographed,\u201d comments Steven. \u201cWe\u2019re continuing to run a huge number of jobs and providing access to all our other data and services, so we have to use every last bit of our capacity to do make the move seamless. Fortunately, funding from the UK Government supports all our data centre activities, so we have installed temporary capacity capable of handling twice as many jobs as usual. We\u2019re confident that things will go pretty smoothly.\u201d<\/p>\n\n\n\n<blockquote class=\"vf-blockquote\"><p>We have installed temporary capacity capable of handling twice as many jobs as usual.<\/p><\/blockquote>\n\n\n\n<p>Most of the data is duplicated and currently stored in racks of servers at the two London data centres. To ensure uninterrupted service, the contents are also replicated in the Genome Campus at Hinxton outside Cambridge as the physical hardware is moved to Hemel Hempstead. By the end of 2014, the public will access all EMBL-EBI data through the new Hemel Hempstead data centre.<\/p>\n\n\n\n<p>\u201cOur technical teams have been working with everyone at EMBL-EBI to ensure our migration plans are right, down to the last detail,\u201d says Steven.<\/p>\n\n\n\n<p>The millions of jobs being run every month are supported by EMBL-EBI middleware called Job Dispatcher, which handles utilities for the European Nucleotide Archive, Ensembl Genomes, UniProt, Pfam, PDBe and many others. Job Dispatcher brings analysis functionality for most of the institute\u2019s data resources and the EBI Search, and provides the means for users to compare gene or protein sequences to find out how they affect function.<\/p>\n\n\n\n<p>\u201cEMBL-EBI is a regular part of the way lots of different research projects are set up,\u201d explains <a title=\"Rodrigo Lopez, Head of Web Development, EMBL-EBI\" href=\"http:\/\/www.ebi.ac.uk\/about\/people\/rodrigo-lopez\" target=\"_blank\" rel=\"noopener noreferrer\">Rodrigo Lopez, head of Web Development at EMBL-EBI<\/a>. \u201cJob Dispatcher helps them establish analytical pipelines to query the data automatically and combine it with their own. It also does plenty for users making individual queries, who are exploring the data in a more organic way.\u201d<\/p>\n\n\n\n<p>The growth in demand for biological data has been dramatic: compared with almost 12 million jobs this September, EMBL-EBI ran an average of 5.4 million jobs per month in 2013, when users could search over 60,000 datasets from life-science experiments.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?<\/p>\n","protected":false},"author":13,"featured_media":2289,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[28,782,36,76],"embl_taxonomy":[],"class_list":["post-2280","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-lab-matters","tag-bioinformatics","tag-database","tag-embl-ebi","tag-service"],"acf":{"article_intro":"<p>How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?<\/p>\n","related_links":[{"link_description":"European Bioinformatics Institute (EMBL-EBI): services","link_url":"http:\/\/www.ebi.ac.uk\/services"},{"link_description":"","link_url":""}],"article_sources":[{"source_description":"<p>Goujon, McWilliam, Li,\u00a0<em>et al.<\/em> <em>Nucleic Acids Research<\/em>, 1 July\u00a02010. DOI: 10.1093\/nar\/gkq313<\/p>\n","source_link_url":"http:\/\/europepmc.org\/abstract\/MED\/20439314"}],"vf_locked":false,"featured":false,"color":"#007B53"},"embl_taxonomy_terms":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Job Dispatched - EMBL-EBI<\/title>\n<meta name=\"description\" content=\"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Job Dispatched - EMBL-EBI\" \/>\n<meta property=\"og:description\" content=\"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\" \/>\n<meta property=\"og:site_name\" content=\"EMBL\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/embl.org\/\" \/>\n<meta property=\"article:author\" content=\"www.facebook.com\/EMBLEBI\" \/>\n<meta property=\"article:published_time\" content=\"2014-10-21T09:26:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-07-13T10:01:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"620\" \/>\n\t<meta property=\"og:image:height\" content=\"380\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Mary Todd Bergman\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@EMBLEBI\" \/>\n<meta name=\"twitter:site\" content=\"@embl\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mary Todd Bergman\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"NewsArticle\",\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\"},\"author\":{\"name\":\"Mary Todd Bergman\",\"@id\":\"https:\/\/www.embl.org\/news\/#\/schema\/person\/052a43af2beb3860bffa67c0f0474875\"},\"headline\":\"Job dispatched\",\"datePublished\":\"2014-10-21T09:26:25+00:00\",\"dateModified\":\"2021-07-13T10:01:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\"},\"wordCount\":431,\"publisher\":{\"@id\":\"https:\/\/www.embl.org\/news\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png\",\"keywords\":[\"bioinformatics\",\"database\",\"embl-ebi\",\"service\"],\"articleSection\":[\"Lab Matters\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\",\"url\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\",\"name\":\"Job Dispatched - EMBL-EBI\",\"isPartOf\":{\"@id\":\"https:\/\/www.embl.org\/news\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png\",\"datePublished\":\"2014-10-21T09:26:25+00:00\",\"dateModified\":\"2021-07-13T10:01:00+00:00\",\"description\":\"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage\",\"url\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png\",\"contentUrl\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png\",\"width\":620,\"height\":380,\"caption\":\"Snapshot of EMBL-EBI users, 20 October 2014 at 13:55. The data hosted at EMBL-EBI is queried by users all over the world, 24\/7. IMAGE: EMBL-EBI\"},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.embl.org\/news\/#website\",\"url\":\"https:\/\/www.embl.org\/news\/\",\"name\":\"European Molecular Biology Laboratory News\",\"description\":\"News from the European Molecular Biology Laboratory\",\"publisher\":{\"@id\":\"https:\/\/www.embl.org\/news\/#organization\"},\"alternateName\":\"EMBL News\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.embl.org\/news\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.embl.org\/news\/#organization\",\"name\":\"European Molecular Biology Laboratory\",\"alternateName\":\"EMBL\",\"url\":\"https:\/\/www.embl.org\/news\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.embl.org\/news\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2025\/09\/EMBL_logo_colour-1-300x144-1.png\",\"contentUrl\":\"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2025\/09\/EMBL_logo_colour-1-300x144-1.png\",\"width\":300,\"height\":144,\"caption\":\"European Molecular Biology Laboratory\"},\"image\":{\"@id\":\"https:\/\/www.embl.org\/news\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/embl.org\/\",\"https:\/\/x.com\/embl\",\"https:\/\/www.instagram.com\/embl_org\/\",\"https:\/\/www.linkedin.com\/company\/15813\/\",\"https:\/\/www.youtube.com\/user\/emblmedia\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.embl.org\/news\/#\/schema\/person\/052a43af2beb3860bffa67c0f0474875\",\"name\":\"Mary Todd Bergman\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.embl.org\/news\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/61b3547fb0d7b56f0de2f0800bbfe2be156d2497f70f754bde9028f71fd85683?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/61b3547fb0d7b56f0de2f0800bbfe2be156d2497f70f754bde9028f71fd85683?s=96&d=mm&r=g\",\"caption\":\"Mary Todd Bergman\"},\"description\":\"Mary Todd Bergman is the Senior Communications Officer at EMBL-EBI. She is surrounded by people who have a calling and love what they do, and helps them spread the word about it. Who could ask for more?\",\"sameAs\":[\"http:\/\/www.ebi.ac.uk\",\"www.facebook.com\/EMBLEBI\",\"https:\/\/x.com\/EMBLEBI\"],\"url\":\"https:\/\/www.embl.org\/news\/author\/mary\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Job Dispatched - EMBL-EBI","description":"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/","og_locale":"en_US","og_type":"article","og_title":"Job Dispatched - EMBL-EBI","og_description":"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?","og_url":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/","og_site_name":"EMBL","article_publisher":"https:\/\/www.facebook.com\/embl.org\/","article_author":"www.facebook.com\/EMBLEBI","article_published_time":"2014-10-21T09:26:25+00:00","article_modified_time":"2021-07-13T10:01:00+00:00","og_image":[{"width":620,"height":380,"url":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","type":"image\/png"}],"author":"Mary Todd Bergman","twitter_card":"summary_large_image","twitter_creator":"@EMBLEBI","twitter_site":"@embl","twitter_misc":{"Written by":"Mary Todd Bergman","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#article","isPartOf":{"@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/"},"author":{"name":"Mary Todd Bergman","@id":"https:\/\/www.embl.org\/news\/#\/schema\/person\/052a43af2beb3860bffa67c0f0474875"},"headline":"Job dispatched","datePublished":"2014-10-21T09:26:25+00:00","dateModified":"2021-07-13T10:01:00+00:00","mainEntityOfPage":{"@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/"},"wordCount":431,"publisher":{"@id":"https:\/\/www.embl.org\/news\/#organization"},"image":{"@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage"},"thumbnailUrl":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","keywords":["bioinformatics","database","embl-ebi","service"],"articleSection":["Lab Matters"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/","url":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/","name":"Job Dispatched - EMBL-EBI","isPartOf":{"@id":"https:\/\/www.embl.org\/news\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage"},"image":{"@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage"},"thumbnailUrl":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","datePublished":"2014-10-21T09:26:25+00:00","dateModified":"2021-07-13T10:01:00+00:00","description":"How does EMBL-EBI run millions of jobs for its users while moving its two large data centres?","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.embl.org\/news\/lab-matters\/1410_data-move\/#primaryimage","url":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","contentUrl":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","width":620,"height":380,"caption":"Snapshot of EMBL-EBI users, 20 October 2014 at 13:55. The data hosted at EMBL-EBI is queried by users all over the world, 24\/7. IMAGE: EMBL-EBI"},{"@type":"WebSite","@id":"https:\/\/www.embl.org\/news\/#website","url":"https:\/\/www.embl.org\/news\/","name":"European Molecular Biology Laboratory News","description":"News from the European Molecular Biology Laboratory","publisher":{"@id":"https:\/\/www.embl.org\/news\/#organization"},"alternateName":"EMBL News","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.embl.org\/news\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.embl.org\/news\/#organization","name":"European Molecular Biology Laboratory","alternateName":"EMBL","url":"https:\/\/www.embl.org\/news\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.embl.org\/news\/#\/schema\/logo\/image\/","url":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2025\/09\/EMBL_logo_colour-1-300x144-1.png","contentUrl":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2025\/09\/EMBL_logo_colour-1-300x144-1.png","width":300,"height":144,"caption":"European Molecular Biology Laboratory"},"image":{"@id":"https:\/\/www.embl.org\/news\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/embl.org\/","https:\/\/x.com\/embl","https:\/\/www.instagram.com\/embl_org\/","https:\/\/www.linkedin.com\/company\/15813\/","https:\/\/www.youtube.com\/user\/emblmedia\/"]},{"@type":"Person","@id":"https:\/\/www.embl.org\/news\/#\/schema\/person\/052a43af2beb3860bffa67c0f0474875","name":"Mary Todd Bergman","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.embl.org\/news\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/61b3547fb0d7b56f0de2f0800bbfe2be156d2497f70f754bde9028f71fd85683?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/61b3547fb0d7b56f0de2f0800bbfe2be156d2497f70f754bde9028f71fd85683?s=96&d=mm&r=g","caption":"Mary Todd Bergman"},"description":"Mary Todd Bergman is the Senior Communications Officer at EMBL-EBI. She is surrounded by people who have a calling and love what they do, and helps them spread the word about it. Who could ask for more?","sameAs":["http:\/\/www.ebi.ac.uk","www.facebook.com\/EMBLEBI","https:\/\/x.com\/EMBLEBI"],"url":"https:\/\/www.embl.org\/news\/author\/mary\/"}]}},"field_target_display":"embl","field_article_language":{"value":"english","label":"English"},"fimg_url":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","featured_image_src":"https:\/\/www.embl.org\/news\/wp-content\/uploads\/2014\/10\/Job_dispatched_EMBL-EBI_users-1.png","_links":{"self":[{"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/posts\/2280","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/comments?post=2280"}],"version-history":[{"count":10,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/posts\/2280\/revisions"}],"predecessor-version":[{"id":19571,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/posts\/2280\/revisions\/19571"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/media\/2289"}],"wp:attachment":[{"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/media?parent=2280"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/categories?post=2280"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/tags?post=2280"},{"taxonomy":"embl_taxonomy","embeddable":true,"href":"https:\/\/www.embl.org\/news\/wp-json\/wp\/v2\/embl_taxonomy?post=2280"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}