收藏 分享(赏)

the athena startup kit.pdf

上传人:weiwoduzun 文档编号:1754346 上传时间:2018-08-22 格式:PDF 页数:5 大小:101.96KB
下载 相关 举报
the athena startup kit.pdf_第1页
第1页 / 共5页
the athena startup kit.pdf_第2页
第2页 / 共5页
the athena startup kit.pdf_第3页
第3页 / 共5页
the athena startup kit.pdf_第4页
第4页 / 共5页
the athena startup kit.pdf_第5页
第5页 / 共5页
亲,该文档总共5页,全部预览完了,如果喜欢就下载吧!
资源描述

1、The Athena Startup KitW. T. L. P. LavrijsenLBNL, Berkeley, CA 94720, USAThe Athena Startup Kit (ask), is an interactive front-end to the Atlas software framework (athena). Written inpython, a very eective “glue” language, it is build on top of the, in principle unrelated, code repository, build,conf

2、iguration, debug, binding, and analysis tools. Ask automates many error-prone tasks that are otherwise leftto the end-user, thereby pre-empting a whole category of potential problems. Through the existing tools, whichask will setup for the user if and as needed, it locates available resources, maint

3、ains job coherency, manages therun-time environment, allows for interactivity and debugging, and provides standalone execution scripts. Anend-user who wants to run her own analysis algorithms within the standard environment can let ask generatethe appropriate skeleton package, the needed dependencie

4、s and run-time, as well as a default job options script.For new and casual users, ask comes with a graphical user interface; for advanced users, ask has a scriptablecommand line interface. Both are built on top of the same set of libraries. Ask does not need to be, and isnt,experiment neutral. Thus

5、it has built-in workarounds for known gotchas, that would otherwise be a majortime-sink for each and every new user. Ask minimizes the overhead for those physicists in Atlas who just wantto write and run their analysis code.1. INTRODUCTIONAtlas 1 is a multi-purpose high energy physicsexperiment plan

6、ned for the Large Hadron Col-lider 2 (LHC), which is scheduled for startup after2007. The data processing needs of Atlas, its varietyin subsystems, and its expected longevity pose a bigchallenge for the software that will be needed for ex-tracting physics out of the future Atlas data. Thischallenge

7、has attracted a large software community,with developers working today, still a few year beforethe experiment starts running, on every aspect of thesystem. Most of the huge Atlas code base, from thesoftware that the end-user runs to the configurationof the infrastructure tools, is currently under he

8、avydevelopment. This has resulted in a state of flux, as aconsequence of which all software usage becomes low-level, and all users end up fixing the same problemsmultiple times. The threshold for new users is high,and casual users are, time and again, forced to relearnmost of what they thought they

9、knew.A large fraction of the Atlas software runs withinthe athena framework, and in order to alleviate sev-eral of the chores that developers and end-users alikehave to perform in order to work with that system,the Athena Startup Kit (ask) has been developed.Its main features are: Automation of end-

10、user tasks, to save time. Integration of the framework, the releases, andthe release tools to improve perceived coherency. Encapsulation of volatile wisdom (from web-pages and peoples heads) of known problemsand their workarounds, to make it more accessi-ble.There are several dierent categories of a

11、thenausers and ask addresses the needs of all of them.Ask performs the steps needed to setup tools, lo-cate resources, work around known problems, and ex-ecute the athena program. In addition, ask hasathena specific functionality: it can generate andupdate user code meant for athena execution (“algo

12、-rithms”), generate default job options scripts, and up-date the dependencies for dierent releases and com-pile modes.A simplistic implementation of ask would meanthat its maintenance would turn into a full-time job,which is unacceptable. Thus, ask is coded in sucha way that all expected changes are

13、 properly takeninto account when they occur, that a solution thathas been successfully applied before is applied againif the problem resurfaces, and that it has a “Plan B”for most tasks. This paper describes in detail how thisrobustness is achieved.2. WORK MODELThere are three main aspects to everyt

14、hing that askperforms: first, setup a clean-room to work in; then,decompose the task at hand; and finally, always beready to handle failure.2.1. The Clean-RoomThere is a whole score of problems related to im-proper configuration of the working environment bythe end-user. In order to keep full contro

15、l of the en-vironment, ask does not require any actions outsideits own process (or any of its subprocesses). Althoughthe user needs to have full access to the environmentin case all else fails, in general she is shielded. Askaccomplishes this by executing a working shell in thebackground, to which i

16、t establishes open pipes for in-put and output, and it then works on this shell as ifComputing in High Energy and Nuclear Physics, 24-28 March 2003, La Jolla, California1ePrint cs.SE/0306083TUJT002it was itself just another end-user. This shell is its“clean-room,” and ask has full control: all commu

17、ni-cation proceeds through an API, which allows ask toverify and validate any commands, their progress, andtheir results. It is not uncommon to allow commandsthat may cause havoc, and simply correct settings af-terwards.A shell that is started from the current process in-herits the environment and c

18、onfiguration of the cur-rent process, which can be invalid to begin with. Notehowever, that what may seem like an invalid config-uration to ask may in fact be an experimental setupfrom an expert user, who really wants to have the envi-ronment setup this way in order to see what happens.Therefore, as

19、k can not simply override any existingconfiguration. Instead, it accepts the settings fromthe user and only provides what appears to be miss-ing. This is a very productive and pleasant way ofworking: as an expert user, you configure only thatpart of the system that you are interested in, and asktake

20、s care of the rest.In practice, it turns out that for beginning and ca-sual users a dierent approach is needed: many ofthem have residual configuration in their login files.This usually does not cause any problems at first, butsooner or later these settings will go out of date, leav-ing the user puz

21、zled when things start to mysteriouslyfail. The user will, typically, blame1this on ask andmay decide to drop the tool in disappointment. Con-sequently, ask comes in two modes: expert and non-expert (the latter is the default). In non-expert mode,ask removes any known Atlas software configurationfro

22、m the environment before executing any of its owncode. This way, ask will operate as if it started froma clean environment and perform the whole setup andconfiguration itself.Regardless of whether the user or ask has done theconfiguration, every task is executed from a knownstate: ask enters a base

23、directory, from where thetask can move into its working directory, and it locatesthe required resources and verifies their configuration,repairing the setup as needed (taking guidelines fromexperts, overriding non-experts). Each task can thenbe implemented in a straightforward way, since it will,by

24、design, not get executed until it has a very goodchance of succeeding.If an environment setup is irreparably broken, or ifa critical resource is unlocatable, ask has no choicebut to stop. However, since such a problem occurs invery specific cases, it is often (but not always) possibleto present a cl

25、ear and concise error report to the user.See Section 2.3 for a more detailed discussion of errorhandling by ask.1In fact, it used to be that every ask bug report was resolvedby simply having the user clean up his environment.2.2. Task DecompositionMany of the tasks performed by ask use the sametools

26、 and require similar resources. Further, stepswithin individual tasks sometimes overlap and canconsequently be shared. In fact, most tasks sharethe same initialization and finalization step. Thus,it makes sense to implement the tasks in a layeredexecution model: Encapsulate toolsEach task has access

27、 to the execution shell, butonly standard unix commands are executed di-rectly. Tools that are part of Atlas software,however, are normally2used directly. Instead,tools are encapsulated in modules, which en-ables ask to locate tools and to verify their setupand possibly update or repair it, before a

28、ny at-tempt is made to perform actions with the tool.This is the lowest layer. Refactorize stepsIt is a good programming practice to minimizethe amount of duplicate code and in ask thismeans that all steps that are shared betweentasks are refactored. Further, the order of ex-ecution of steps is also

29、 often the same. By de-sign many actions need the same set of param-eters, and it is therefore straightforward to usemeta-programming to create templates that areparametrized on the core action of a task. Thisguarantees that the execution order is proper,which greatly simplifies the implementation o

30、findividual steps. This is the programmaticlayer. Construct recipesThe end-user typically does not think abouttasks in the same granularity as tool authorsdo. Thus, recipes, that are end-user tasks,are constructed from the programmatic tasks.Since programmatic tasks are supposed to beself-contained,

31、 they can be freely mixed andmatched in the recipes: it is only the goal ofthe recipes that puts constraints on their order.This is the user layer. User interfacesThe recipes typically take input and have out-put. The former needs to be collected fromthe user, and the latter needs to be presentedbac

32、k to the user. The user interfaces connect in-put/output elements with recipes to present the2Similar to the setup of the environment, the user can accessthe tools directly if all else failsComputing in High Energy and Nuclear Physics, 24-28 March 2003, La Jolla, California2ePrint cs.SE/0306083TUJT0

33、02ask functionality to the user. There is a graphi-cal interface and a command line interface. Thisis the application layer.Third party programs or scripts, can access askfunctionality at any layer, but do so most commonlyat the programmatic and/or user layer. These pro-grams are typically similar i

34、n setup as the user inter-faces (in eect, they are also located in the applicationlayer). For example, the ask distribution includes aprogram that performs full Atlas software builds andis targeted at software librarians. But in principle,it is just a batch version of the command line inter-face wit

35、h a few dierent defaults and a few extendedrecipes.2.3. Error HandlingThere are several dierent kinds of errors that askcan encounter: Random.Errors due to (temporarily) missing resourcessuch as disk access, kerberos tokens, the tagcollector, etc. These errors come and go and, un-fortunately, the so

36、lution is usually “come backlater, and try again” or requires some user in-tervention. Reporting is often, but not always,straightforward. For example, if a kerberosticket is missing and ask attempts to access afile in the code repository, it will be cvs thatfails and ask will only have an error fro

37、m thattool to report, thus masking the real error. User actions.Errors because of a file that does not compile, anexecutable that does not link, an option scriptthat the user requested but that does not ex-ist, etc. Reporting these errors is simple: askneeds to retrieve the error code, stop the task

38、,and report the error message from the tool thatfailed. Broken system.Errors as a result of misconfiguration of the At-las software distribution that is used. There isnot much point in reporting this kind of error,unless the user is testing the distribution. In-stead, ask attempts to work around the

39、 prob-lem. If that fails, then the requested task cannot be performed.It is the last item that is the most interesting, be-cause in handling that category of errors there is themost to gain in terms of productivity for the end-user,especially if he or she is a novice or casual user. Inorder to deal

40、with broken systems, ask has severaloptions. First of all, it may come prepared: for sev-eral releases and configurations, it comes with specificworkarounds that are simply programmed into ask.That way, the problem will not appear in the firstplace, because the workaround is the default.It gets more

41、 interesting when a task does fail. Inthat case, ask will try a few more things that mightwork, such as locating an alternative resource, query-ing an alternative source, or simply accepting a defaultand hope for the best. The ticket here is that this isprecisely how an end-user would attempt to tac

42、kle theproblem herself, with the exception that ask can per-form the trials much, much faster, and it has thereforea good chance of arriving at a solution with the end-user never knowing that there was a problem.As a final resort, ask sometimes explicitly fixes aworkspace. Since this usually involve

43、s moving or re-placing files and/or directories, this is done only whenthe case is very clear.Note that as before, when all else fails, ask doesallow expert access at all levels, at any time.3. IMPLEMENTATIONOne of the stated goals of ask (see section 1)isthe integration of the framework, the releas

44、es, andthe release tools. The natural choice is then to imple-ment ask in a scripting language, because that is theonly way to eectively and productively integrate sucha variety of tools and environments. The prevalentscripting language in Atlas software is python 3,which is e.g. used for the script

45、ing facilities of theathena framework. Python is often praised for itseective use as a “glue” language, because of its ex-tending and embedding features, and the choice forpython is therefore natural.Python is portable, interpreted, and gives you alot of functionality per line of code because of its

46、 ex-tensive set of ready-to-use modules that come withevery standard installation. That, and the fact thatmost (if not all) unix distributions actually ship witha version of python, means that all the user needsto do to install ask, is to make sure that the scriptsare located in a directory that is

47、in included in theirPATH environment variable. This is the case for allAtlas accounts at CERN, because ask has been in-stalled in the ocial scripts directory. Consequently,all Atlas users have immediate access to ask withoutthe need of any further setup.4. USER INTERFACEThis section describes how th

48、e ask functionality ispresented to the user. The details of the individualcommands, buttons etc. are explained in the man-ual 4, which is included with the distribution, andare not further elaborated here.Computing in High Energy and Nuclear Physics, 24-28 March 2003, La Jolla, California3ePrint cs.

49、SE/0306083TUJT002There are two user interfaces in ask: a graphicaluser interface (GUI) and a command line interface(CLI). The former is meant for beginning and casualusers, as well as for tutorials; whereas the latter ismore attuned to advanced users, developers, and li-brarians.4.1. Graphical User InterfaceHistorically, the first usage of ask was to gener-ate an athena skeleton package. In order to beable to capture all user-configurable parameters atonce, a small graphical interface interface was devel-oped. This proved especially practical for packageupdates,

展开阅读全文
相关资源
猜你喜欢
相关搜索

当前位置:首页 > 企业管理 > 经营企划

本站链接:文库   一言   我酷   合作


客服QQ:2549714901微博号:道客多多官方知乎号:道客多多

经营许可证编号: 粤ICP备2021046453号世界地图

道客多多©版权所有2020-2025营业执照举报