• Minimum partition size on Mira is 512 nodes; maximum backfill is 8192 nodes

  Mar 4th 2013

  By: Tim Williams

  No comments

  As has been explained in a recent email to Mira users, the minimum partition you can use on the machine is 512 nodes. If you request fewer nodes, you still pay from your allocation for all 512, and the unused nodes are idle. On Cetus, the minimum partition size is 128 nodes.

  As some of you exhaust your ESP allocations on Mira, you will notice your jobs going into the “backfill” queue. These are queued with low priority relative to positive-allocation-balance jobs, but will run if resources are available and no normal jobs are available to fit the space. The maximim size job allowed in backfill mode is 8192 nodes.

• Please test your codes with new system driver on Vesta

  Jan 31st 2013

  By: Tim Williams

  No comments

  This coming Monday and Tuesday (4-5 Feb. 2013), Vesta will be down for extended maintenance, to install the latest BG/Q system driver from IBM (V1R2M0). Eventually, this driver version will be installed on Mira. Please help ALCF and yourselves by building and testing your Early Science codes on Vesta after the upgrade, if you can. Let us know if something breaks.

• Cetus Down

  Jan 9th 2013

  By: Tim Williams

  No comments

  There will be an official notice going out soon, but be aware that Cetus is down and will be down for a number of days. This is related to the Vesta downtime—the BG/Q rack that’s currently designated as Cetus is being combined with Vesta to make Vesta a 2-rack system. We have a new rack that will be designated as Cetus. My best estimate is 5 days of downtime for Cetus (yesterday’s notice to vesta-notify and mira-notify lists estimated 5 days downtime for Vesta).

• Early Science on Mira is on; time allocations in place

  Dec 19th 2012

  By: Tim Williams

  No comments

  The Early Science period is officially underway. Mira came back online after acceptance testing on the evening of Monday 17 December. After an initial glitch in setting up the computer time allocations for the ESP projects, the correct allocations are now in place. These are what you were awarded as target allocation when your project was selected for the ESP. On Mira, the command

          cbank-list-allocations -u yourUserName -r mira

  will show you the amout and usage of your allocation.

  Our one-rack test and development machine, Cetus (cetus.alcf.anl.gov) is now also available to ESP users.

  The Early Science period should last through mid-March. When there is concrete information about the exact transition date, I’ll send out an email with the date and information about how the transition to production usage will impact the Early Science projects. You should have used up your ESP project allocations by then.

• Allocations for *_esp projects are active

  Oct 22nd 2012

  By: Tim Williams

  No comments

  Please note that, since the 24 accepted racks of Mira were turned over to Early Science, time allocations for the ESP projects have been in place. The amounts of the allocations are not yet correct; all were set to a placeholder value of 50 million core hours. When ALCF and Mira are ready, we will establish the formal allocations for the projects. These allocations will be based on the target awards from the letters informing you of your Early Science Program awards. Those are:

  PROJECT               AWARD (in millions of Mira core-hours)
  --------------------  --------------------------------------
  GFDL_esp              150
  Mat_Design_esp         50
  Autoignition_esp      150
  Bulk_Properties_esp   150
  DarkUniverse_esp      150
  MADNESS_MPQC_esp      150
  CFDAnisotropic_esp    150
  GroundMotion_esp      150
  HSCD_esp              150
  TurbNuclComb_esp      150
  LatticeQCD_esp        150
  TurbChannelFlow_esp    60
  AbInitioC12_esp       110
  NAMD_esp               80
  PlasmaMicroturb_esp    50
  MultiscaleMolSim_esp  150

• Mira access added; usage announcement mailed out

  Oct 11th 2012

  By: Tim Williams

  No comments

  Those of you who responded to the email asking for new ESP users that need to get access to Mira for Early Science runs on the accepted 24 racks of the machine: you should all now have access to Mira (and Vesta), except possibly those just now getting an ALCF account for the first time (you have additional application procedures to do, for which you should’ve received instructions).

  Today I sent out an email to mira-early-users with some details about using Mira between now and the start of the 48-rack acceptance testing (around mid November). For those of you in the Early Science Program projects, you can now run jobs up to 16 racks in the “ESP” queue. For 24 racks and higher, you will land in the “ESP-bigrun” queue, which will be manually managed. This should mainly be for scaling tests, not scientific runs, since when you run on 24 racks and higher you’ll be using some of the unaccepted Mira nodes, and can’t expect reliability as you get on the 24 accepted racks (where all your jobs of 16 racks and less will run).

  Remember that on BG/Q, 1 rack is 1024 nodes (same as BG/P), but is 16K cores (as opposed to 4K cores on BG/P).

• Mira access for general ESP users—send names this week

  Oct 4th 2012

  By: Tim Williams

  No comments

  A reminder to send me information about additional from your ESP project teams that you’d like to be able to access Mira starting next week.

  In order to get on the list I’ll be handing off to User Services, you need to send in the information by the end of this week (tomorrow, Friday 4 Oct). Send to earlyscience@alcf.anl.gov .

  If you already have Mira access, you don’t need to do anything. The information I need to get someone Mira access is:

  • First, Last name.
  • ALCF username
  • email address
  • ESP project (short name, from list below)

  PI                      Project Short Name
  Venkatramani Balaji     GFDL_esp
  Larry Curtiss           Mat_Design_esp
  Christos Frouzakis      Autoignition_esp
  Mark Gordon             Bulk_Properties_esp
  Salman Habib            DarkUniverse_esp
  Robert Harrison         MADNESS_MPQC_esp
  Kenneth Jansen          CFDAnisotropic_esp
  Thomas Jordan           GroundMotion_esp
  Alexei Khokhlov         HSCD_esp
  Don Lamb                TurbNuclComb_esp
  Paul Mackenzie          LatticeQCD_esp
  Robert Moser            TurbChannelFlow_esp
  Steven C Pieper         AbInitioC12_esp
  Benoit Roux             NAMD_esp
  William Tang            PlasmaMicroturb_esp
  Gregory Voth            MultiscaleMolSim_esp

• ESV two-week Mira access now estimated 2nd 2 weeks of August

  Jul 3rd 2012

  By: Tim Williams

  No comments

  The current revised estimate for the two-week period when ESP projects will have acces to the full Mira for testing codes at scale (ESV period), is the last half of August, possibly starting around August 20. Please plan for this. (I still haven’t heard from most of the ESP projects who their 1 or 2 ESV users will be. Please email earlyscience@alcf.anl.gov with this information.

  [Update 10/3/2012: As indicated in a couple of mass emailings to ESP project participants, the plan changed and there will be no specific 2-week ESV period. We have accepted 24 of the 48 racks, and access will be extended starting the week of Oct. 7 to all ESP project participants to run science on those 24 racks.]

• Who do you want to access Mira during the ESV?

  Jun 25th 2012

  By: Tim Williams

  No comments

  So far, I’ve only heard back from 3 projects about the one or two people designated to get access to Mira during the ESV window. Please think it over and send me (earlyscience@alcf.anl.gov) the names of those for your project.

  Here’s a snip from the email of June 4:

  There will be a two-week period (called the Early Science Verification – ESV), prior to the Mira acceptance-test period, when the Early Science codes can be run on the system. Their running correctly is a prerequisite for starting the acceptance testing (assuming the problem is with the system, and not the code or input decks). The ESV is an important milestone for the Early Science Program, the ALCF, and IBM. Please be prepared to exercise your application at scale—ideally running problems comparable with your planned production runs during the Early Science period. This is also a good opportunity for you to get your first multi-rack testing on Mira done.

  Our best estimate for the intended start of acceptance testing is mid August, which places the ESV in the first two weeks of August. We expect to have the login/compile servers ready for your access at least one week prior. Please prepare your codes, and plan for members of your teams to run these tests. Only 1 or 2 key people from each project will have access to Mira in this period, so please identify those people in advance.

  Current estimate is that ESV will start later—second half of August or after.

• Agenda and slides from “Code for Q” workshops

  Jun 4th 2012

  By: Tim Williams

  No comments

  The agendas from the March and April ESP “Code for Q” workshops contain hyperlinks to the slides from the presentations. You can find the agendas here:

  • April ESP “Code for Q” Workshop
  • March ESP “Code for Q” Workshop

• April “Code for Q” Workshop – have you registered yet?

  Apr 10th 2012

  By: Tim Williams

  No comments

  The April ESP “Code for Q” workshop will be held April 30 – May 2, 2012.

  If you plan to attend, please register using instructions emailed to you (if you lost or didn’t receive instructions, email earlyscience@alcf.anl.gov or espworkshop@alcf.anl.gov and we’ll send them to you). Registration deadlines:

  April 16 – Non US citizens (who are not ANL employees)
  April 23 – US citizens and ANL employees

  This workshop will start at 1:00 pm on Monday 30 April, with a full afternoon of formal presentations about Blue Gene/Q hardware and software, and our Very Early Access System 1-rack Blue Gene/Q machine you’ll have access to. Tuesday and Wednesday May 1-2 will be all-day hands-on sessions, with mini-presentations by some tools and libraries teams about their efforts on BG/Q and how to use their software on VEAS if available. Developers from these tools and libraries teams will be available to help you use their software, and ALCF staff will be on hand to help you get your code ported and running on VEAS.

• One-Rack BG/Q System – VEAS

  Mar 22nd 2012

  By: Tim Williams

  No comments

  Developers from the Early Science projects (and Tools & Libraries project) are eligible for accounts on the ALCF’s new one-rack BG/Q system, in its current form as the Very Early Access System (VEAS). If you would like to get access, please email earlyscience@alcf.anl.gov or follow the new instructions for requesting the VEAS resource.. The VEAS is currently living on the test & development rack named cetus.

  (The system software and compilers on the VEAS are pre-production. There have been, are, and will continue to be glitches, unplanned downtimes, and software updates. In exchange for early access, you must agree to be tolerant of this environment.)

• March “Code for Q” Workshop Held

  Mar 22nd 2012

  By: Tim Williams

  No comments

  The ESP March 2012 Workshop, “Code for Q,” was held at the ALCF. We had 57 attendees, with representatives from 13 of the 16 ESP projects and several teams under the Tools and Libraries project. After an afternoon of formal presentations on Monday March 19th, we had two days of mostly hands-on work. By the end of the workshop, several ESP scientific applications and several tools/libraries packages were running on our one-rack BG/Q system, the VEAS (Very Early Access System).

• March “Code for Q” Workshop – have you registered yet?

  Mar 7th 2012

  By: Tim Williams

  No comments

  If you plan to attend the March ESP workshop, please register using instructions emailed to you (if you lost or didn’t receive instructions email earlyscience@alcf.anl.gov or espworkshop@alcf.anl.gov and we’ll send them to you). Registration deadlines:

  • March 5 – Non US citizens (who are not ANL employees)
  • March 12 – US citizens and ANL employees

  If you are a non-US citizen and not an Argonne employee, the registration deadline has passed. You should consider coming to the 2nd workshop, whose dates are now set as April 30 – May 2.

• Upcoming Workshops – March Dates Moved to 19-21

  Feb 6th 2012

  By: Tim Williams

  No comments

  As discussed in a recent email to ESP investigators, the ALCF is planning a couple of workshops for the Early Science Program. The first was tentatively scheduled for March 5-7, but because of a conflict in our conference center is being rescheduled for March 19-21.

  These are meant to be primarily technical, hands-on workshops for informal sharing of information and expertise; ALCF staff will help to get your code better target to Blue Gene/Q. By the time of the first workshop, we should all have access to the new test & development machine (a one-rack system that is currently being built at the ALCF). A practical goal of the workshops it getting your codes built and running on the T&D system. The 2nd workshop is planned for April–dates TBA.

• Notes from SC11 – Blue Gene/Q details public

  Nov 15th 2011

  By: Tim Williams

  No comments

  Last night, IBM made a big announcement about Blue Gene/Q on the exhibit floor at SC11 in Seattle. Today, they are presenting a couple of papers and otherwise publicizing BG/Q. This means the lifting of the nondisclosure requirement for details of BG/Q hardware, software, and performance. I will be giving a presentation at the Argonne booth at SC on Wednesday at 11:30, which will discuss preparing applications for Mira. This will include some early performance numbers from the various ESP projects, and performance numbers for codes used as part of our contract with IBM—many of which are used by ESP projects. I’ll put those slides online at the ESP website (http://esp.alcf.anl.gov) tonight or tomorrow morning.

• IBM Hot Chips/Interconnects presentations about Blue Gene/Q

  Sep 20th 2011

  By: Tim Williams

  No comments

  ESP project investigators should just have received an email containing slides from two presentations IBM gave about Blue Gene/Q technologies this summer—at the Hot Chips and Hot Interconnects Symposiums. IBM’s permission for us to discuss these liberates some useful technical information from NDA restrictions.

• Looking for ESP work to showcase at SC11

  Jun 21st 2011

  By: Tim Williams

  No comments

  The ALCF would like to showcase some Early Science Program projects at our booth at the SC11 conference. Please contact me (directly or via earlyscience@alcf.anl.gov) if you have some ideas, or efforts/results you would like to have shown. Project investigators will probably soon be contacted directly about this.

• Scheduling the next Early Science workshop

  Feb 18th 2011

  By: Tim Williams

  No comments

  The ALCF would like to host a second workshop for the Early Science program participants. The format this time should be a more detailed presentation and exchange of information—a focused presentation or two on topics like hybrid MPI/OpenMP programming, and presentation of performance studies and project experiences to date (based on Blue Gene/P efforts and early Q hardware tests). I’m looking for feedback now about scheduling from project investigators and participants. Would early-to-mid June work with your schedules? Please feel free to post comments here, or email directly to earlyscience@alcf.anl.gov . This will likely be repeated soon in an email to the PIs and co-PIs.

• Any results/efforts underway on your projects?

  Dec 6th 2010

  By: Tim Williams

  No comments

  If you have anything to report on your ESP projects—accomplishments, or efforts currently underway—I would appreciate an email to earlyscience@alcf.anl.gov with relevant bullets. Of particular interest are efforts on the items presented in the “6-month plan” slides at the Kick-Off workshop. (This is for giving DOE some status on the program.)

• ESP Postdoc Job Posting

  Nov 19th 2010

  By: Tim Williams

  No comments

  Please refer any candidates for ESP postdoc appointments to the ANL jobs website to apply:

  http://www.anl.gov/jobsearch/detail.jsp?userreqid=316934+LCF&lsBrowse=POSTDOC

  There is a longer description on the ALCF web site:

  http://www.alcf.anl.gov/esp-postdoc-opportunities

  If you are interested in advertising the position for your ESP project in some appropriate venue for your science area, please let me know via an email earlyscience@alcf.anl.gov .

• Any Additional Institutions Needing AECI Agreement with IBM?

  Nov 10th 2010

  By: Tim Williams

  No comments

  I am compiling a final list of institutions participating in the Early Science projects that do not yet have a signed AECI agreement with IBM (nondisclosure agreement enabling people at the institution to access confidential IBM Blue Gene/Q information.) Here are the remaining institutions in need of agreements that I know of. If your institution is not on this list and you intend to participate in one of the ESP projects, please let me know by emailing earlyscience@alcf.anl.gov .

  • University of Virginia
  • University of Utah
  • MIT
  • University of Illinois Urbana-Champaign
  • Jefferson Lab
  • Columbia University

• Thanks for attending the Kick-Off Workshop

  Oct 22nd 2010

  By: Tim Williams

  No comments

  The Kick-Off Workshop happened this week, Monday-Tuesday at the Argonne TCS Conference Center. Thanks to all who attended, presented, and gave us feedback about the workshop. Thanks to the IBM participants, whose presentations were highlights of the workshop, and thanks also to IBM for co-sponsoring Monday’s reception.

  We are in discussion with IBM about having them email out encrypted PDFs of the IBM confidential presentations to workshop attendees. The slides from the project presentations are now hyperlinked to the agenda on the workshop website, http://http://press.mcs.anl.gov/esp10/agenda . Other presentations without confidential information will be linked in as well.

  One discussion point at the workshop was how to manage collaboration with project members at our own institutions, given that the AECI (nondisclosure) agreements are between IBM and the institution. Most likely this will be managed using a “working-group” structure. I’ll report when that’s been worked out (probably including an email to project investigators).

• Remodeled ESP website

  Sep 28th 2010

  By: Tim Williams

  No comments

  The Early Science web pages have been integrated better with the rest of the ALCF website. From the ALCF page (http://www.alcf.anl.gov), the “Research Collaborations” pull-down menu entry “Early Science Program” brings up the new ESP main page. Individual ESP project descriptions have been moved out into individual web pages, consistent with our INCITE and ALCC project pages.

  The convenience URL http://esp.alcf.anl.gov that many of us have been using to get to the ESP website now points to the new main ESP page.

• No more new institutions can get AECI agreements pre-Workshop

  Sep 24th 2010

  By: Tim Williams

  No comments

  If you are thinking about registering for the Early Science Program Kick-Off Workshop, please note that if your institution does not already have an AECI agreement signed with IBM (or is one of a few institutions with agreements in progress), there will not be time to get this done before the Workshop, so you will not be able to attend. If you want to inquire about whether your institution is covered by or in the process of signing an agreement, email earlyscience@alcf.anl.gov . (AECI is the acronym for IBM’s flavor of nondisclosure agreement.)

  If you are not planning to attend the Workshop, but do intend to work on one of the Early Science Program projects, please notify earlyscience@alcf.anl.gov, indicating your institution (university/lab/company where you work). We need to get IBM a complete list of institutions requesting AECI agreements for ESP work as soon as possible. The number is already getting large; at some point we’ll have to close the door and not accept any additional new institutions. If you’ve already let us know about your institution via email or registering for the Workshop, you don’t need to send anything further.

• Please register for Workshop or let us know of intent (for NDA)

  Sep 15th 2010

  By: Tim Williams

  No comments

  It’s important that you register for the upcoming Kick-Off Workshop, if you plan to attend. If you think you may attend, but don’t want to register yet, send email to earlyscience@alcf.anl.gov to let us know. (If you’ve already let us know of your intent, you don’t need to send another mail.) Most importantly, we need to know your home institution, so we know which institutions will need a nondisclosure agreement. Please also compare your citizenship with the lists in the earlier post on export control, to make sure you won’t have a problem attending because of that.

• ESP Postdocs

  Sep 13th 2010

  By: Tim Williams

  No comments

  As you probably know, the Early Science Program has funding to hire postdocs—about 15 in total. We are soliciting your help in recommending candidates for these postdoctoral appointments. The postdocs will be employed and supervised by the ALCF, and will sit here at Argonne, but it is expected that they may travel to home institutions of ESP project investigators for in-depth collaboration and mentoring. Project PIs or other senior participants are expected to mentor the postdoc working on the project, especially in the scientific research area and numerical methods of the project. The postdocs should be participants in analyzing and publishing results of the calculations run on Mira in the Early Science period. ALCF staff will mentor postdocs on code development, performance evaluation, and optimization for Blue Gene/Q. In supervising postdocs, the ALCF will ensure that the postdocs meet the goals of the Early Science Program, the specific project goals, and the postdocs’ career development goals.

• ESP Kick-Off Workshop Announcement

  Sep 13th 2010

  By: Tim Williams

  No comments

  The official announcement of the ESP Kick-Off Workshop is now out. The web site has the registration form, the draft agenda, and more: http://workshops.alcf.anl.gov/esp10

• RSS Feed for Blog Postings

  Sep 10th 2010

  By: Tim Williams

  No comments

  If you like lots of messages, and want to get messages containing new blog posts as they appear, you can subscribe to various-format feeds of the ESP Blog. For example, the URL for the RSS 2.0 feed is:

  http://esp.alcf.anl.gov/blog/?feed=rss2

  There’s a lot of software out there for reading RSS feeds. Some email programs, for example, allow you to subscribe to feeds, resulting in messages appearing in your mail Inbox when new blog posts go up.

• Export Control and ESP Participants

  Sep 8th 2010

  By: Tim Williams

  No comments

  Participants in the ESP projects must meet U.S. Export Control criteria  in order to have access to Blue Gene/Q information. This includes information to be discussed at the Kick-Off Workshop. The PIs received some email about this; here is the relevant part (note that the Workshop website referred to is not yet live):

  Participants for this workshop must meet U.S. Government Export Control criteria. As a result, individuals from these countries may not attend: Cuba, Iran, North Korea, Sudan, Syria.

  Individuals from these countries are eligible to attend: Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Greece, Ireland, Italy, Japan, Luxembourg, Netherlands, New Zealand, Norway, Portugal, Spain, Sweden, Switzerland, Turkey, United Kingdom, United States (including Resident Aliens, i.e. “Green Card” holders)

  If your country of origin is NOT listed, while it may be possible for your institution to obtain the additional clearance required, it may take significant time (six months or longer). Please note that all non-U.S. citizens must complete the necessary 593 documentation process explained in the registration website

• “Nondisclosure” Agreements with IBM

  Sep 8th 2010

  By: Tim Williams

  No comments

  IBM is in the midst of internal discussions about how to manage all the nondisclosure agreements for ESP project institutions. Until we have further word on this from IBM, ESP projects should not contact IBM directly about agreements as was recommended in months past. I’ll post here as soon as we have word from IBM, which I hope will set us up so that Kick-Off Workshop participants will be covered.

  For those that do not know what this is: Anyone wanting to access early Blue Gene/Q hardware, or access or discuss Blue Gene/Q software such as compilers will have to be covered by an agreement between your institution and IBM. IBM doesn’t call these NDAs, but rather AECPs.

• ESP Kick-Off Workshop

  Sep 8th 2010

  By: Tim Williams

  No comments

  The ALCF Early Science Program Kick-Off Workshop has been scheduled for October 18-19 2010 at Argonne’s TCS Conference Center.

  Who should attend? We encourage a representative from the PIs/co-PIs to attend if available to help with project planning and discussion of postdocs. We also encourage a representative from the project who is deeply involved in code development, to learn about the Blue Gene/Q environment and discuss development strategies. If you will attend, please register as soon as the website becomes available. If you want to informally indicate your plan to attend now, you can email earlyscience@alcf.anl.gov to let us know.

• NOTES & CONTEXT

  • Home
  • About the ESP Blog

• RSS

  • Subscribe to RSS

• ARCHIVES

  • March 2013
  • January 2013
  • December 2012
  • October 2012
  • July 2012
  • June 2012
  • April 2012
  • March 2012
  • February 2012
  • November 2011
  • September 2011
  • June 2011
  • February 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010

U.S. Department of Energy Office of Science | UChicago Argonne LLC | Privacy & Security Notice

Argonne National Laboratory is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLC.