《最新四章资源ppt课件.ppt》由会员分享，可在线阅读，更多相关《最新四章资源ppt课件.ppt（69页珍藏版）》请在taowenge.com淘文阁网|工程机械CAD图纸|机械工程制图|CAD装配图下载|SolidWorks_CaTia_CAD_UG_PROE_设计图分享下载上搜索。
1、Globus与资源规范语言RSLGRAM ArchitectureFrom IBM Redbook SG24-6895-012003: Intro to Grid ComputingGRAM Elements Clients Gatekeeper daemon门户监护进程 Job Manager Global Access to Secondary Storage (GASS)辅助存储全局访问 Dynamically-Updated Request Online Coallocator (DUROC)动态更新请求在线协同分配器 User Resource Specification Langu
2、age (RSL)GRAM Clients Three clients:globusrunglobus-job-runglobus-job-submitGRAM管理流程图示Client APIJob ManagerScheduler Specific PluginJob ProcessGatekeeperJob RequestJob cancelstate change callbackfork/su/execfork/exec/waitspsubmit/spqcondor,lsfgatekeeper的作用 gatekeeper：A process, running as root, whic
3、h begins the process of handling allocation requestsperforming mutual authentication of user and resource,determining a local user name for the remote user,starting a job manager which executes as that local user and actually handles the request. In order to start the job manager, the gatekeeper mus
4、t run as a privileged program相关名词解释Resource An entity capable of running one or more processes on behalf of a userClient The process that is using the resource allocation client-side APIJob A process or set of processes resulting from a job request. Job Request A request to gatekeeper to create one
5、or more job processes, expressed in the supplied Resource Specification Language.Job Manager One job manager is created by the gatekeeper to fulfill every request submitted to the gatekeeper.GRAM调度与状态转换模型对各个阶段的解释Unsubmitted ：The job has not yet been submitted to the schedulerStageIn ：The job manager
6、 is staging executable, input, or data files to the jobPending ：The job has been submitted to the scheduler, but resources have not yet been allocated for the job.Active ：The job has received all of its resources, and the application is executingSuspended ：The job has been stopped temporarily by the
7、 schedulerStageOut ：The job manager is staging output files from the job manager host to remote storage. Done ：The job completed successfully.Failed ：The job terminated before completion, as a result of an error, or a user or system cancel.GRAM ComponentsGlobus SecurityInfrastructureJob ManagerGRAM
8、client API calls to request resource allocationand process creation.MDS client API callsto locate resourcesQuery current statusof resourceCreateRSL LibraryParseRequestAllocate &create processesProcessProcessProcessMonitor &controlSite boundaryClientMDS: Grid Index Info ServerGatekeeperMDS: Grid Reso
9、urce Info ServerLocal Resource ManagerMDS client API callsto get resource infoGRAM client API statechange callbacksDUROC（Dynamically-Updated Request Online Co-allocator） Simultaneous allocation of a resource set Handled via optimistic co-allocation based on free nodes or queue prediction advance res
10、ervations will also be supported globusrun will co-allocate specific multi-requests using DUROCGRAM ExamplesThe globus-job-run client is a sample GRAM client, using command-line arguments rather than RSL.% globus-job-run pitcairn.mcs.anl.gov /bin/ls% globus-job-run pitcairn.mcs.anl.gov s myprog% glo
11、bus-job-run pitcairn.mcs.anl.gov s myprog stdin s in.txt stdout s out.txt GRAM ExamplesThe globusrun client is a more involved prototype that allows complicated RSL expressions.% globusrun r pitcairn.mcs.anl.gov f myjob.rsl% globusrun r pitcairn.mcs.anl.gov &(executable=myprog)Resource Management AP
12、Is Globus Toolkit has APIs for RSL, GRAM, and DUROC: globus_rsl globus_gram_client globus_gram_myjob globus_duroc_control globus_duroc_runtimeResource Specification Language 可以用于说明作业要求的通用语言 RSL是GRAM的核心部分，它提供了不同组件之间交换信息的手段，比如应用与资源代理之间，资源协同分配与资源管理之间的信息交换 形式 (attribute=value) 需要GRAM理解这些属性attribute Glob
13、us提供使用RSL的API 可以用于以上之外的更多场合RSL的一些属性 (executable=string) Program to run A file path (absolute or relative) or URL (directory=string) Directory in which to run (default is $HOME) (arguments=arg1 arg2 arg3.) List of string arguments to program (environment=(E1 v1)(E2 v2) List of environment variable na
14、me/value pairsRSL的一些属性(stdin=string) Stdin for program A file path (absolute or relative) or URL(stdout=string) Stdout for program A file path (absolute or relative) or URL(stderr=string) Stdout for program A file path (absolute or relative) or URL(count=integer) Number of processes to run (default
15、is 1)(hostCount=integer) On SMP multi-computers, number of nodes to distribute the “count” processes across(project=string) Project (account) against which to charge(queue=string) Queue into which to submit jobRSL的一些属性 (maxTime=integer) Maximum wall clock or cpu runtime (schedulerss choice) in minut
16、es (maxWallTime=integer) Maximum wall clock runtime in minutes (maxCpuTime=integer) Maximum CPU runtime in minutes (maxMemory=integer) Maximum amount of memory for each process in megabytes (minMemory=integer) Minimum amount of memory for each process in megabytesRSL Attributes For GRAM (jobType=val
17、ue) Value is one of “mpi”, “single”, “multiple”, or “condor” mpi: Run the program using “mpirun -np ” single: Only run a single instance of the program, and let the program start the other count-1 processes. multiple: Start instances of the program using the appropriate scheduler mechanism condor: S
18、tart a Condor processes running in “standard universe”RSL Attributes for GRAM (gramMyjob=value) Value is one of “collective”, “independent” Defines how the globus_gram_myjob library will operate on the processes collective: Treat all processes as part of a single job independent: Treat each of the p
19、rocesses as an independent uniprocessor job (dryRun=true) Do not actually run jobRSL 的替代符 RSL supports simple variable substitutions Substitutions are declared using a list of pairs (rslSubstitution=(SUB1 val1)(SUB2 val2) A substitution is invoked with $(SUB) Processing order: Within scope, processe
20、d left-to-right, Outer scope processed before inner scope Variable definition can reference previously defined variables替代符示例 This&(rslSubstitution=(URLBASE “ftp:/host:1234”) (rslSubstitution=(URLDIR $(URLBASE)/dir) (executable=$(URLDIR)/myfile) is equivalent to this&(executable=ftp:/host:1234/dir/m
21、yfile)GRAM Defined RSL Substitutions GRAM defines a set of RSL substitutions before processing the job request Machine Information GLOBUS_HOST_MANUFACTURER GLOBUS_HOST_CPUTYPE GLOBUS_HOST_OSNAME GLOBUS_HOST_OSVERSIONGRAM Defined RSL Substitutions Paths to Globus GLOBUS_INSTALL_PATH GLOBUS_TOOLS_PATH
22、 GLOBUS_SERVICES_PATH GLOBUS_DEPLOY_PATH Miscellaneous HOME LOGNAME GLOBUS_ID用于DUROC的RSL属性 (subjobStartType=value) Alters the startup barrier mechanism values are “strict-barrier”, “loose-barrier”, “no-barrier” (subjobCommsType=value) values are “blocking-join” and “independent” if value is set to “
23、independent”, the subjob wont be seen from the other subjobs when doing inter-subjob communication. (label=string) Identifier for this subjob (resourceManagerContact=string)(resourceManagerName=string) Resource manager to which to submit a subjobExample: (single resource for now)$ globusrun -r chi/j
24、obmanager-pbs & (executable=/home/abose/test.exe) (host_count=2) (count=4) (arguments=“-t 100 f out.dat) (email_address=“aboseumich.edu) (queue=cac) (pbs_stagein=“morpheus:/home/abose/test.exe) (pbs_stageout=“morpheus:/home/abose/out.dat) (pbs_stdout=/tmp/stdout) (pbs_stderr=/tmp/stderr) (maxwalltim
25、e=10)(jobtype=mpi”)“get test.exe from morpheus and run it on hypnos” - submitted by Globus gatekeeper on chi using PBS job managerRSL Example Resulting PBS Submission Script on Hypnos:#! /bin/sh# PBS batch job script built by Globus job manager#PBS -S /bin/sh#PBS -M aboseumich.edu#PBS -m n#PBS -q ca
26、c#PBS -W stagein=/home/abose/test.exemorpheus.engin.umich.edu:/home/abose/test.exe#PBS -W stageout=/home/abose/out.datmorpheus.engin.umich.edu:/home/abose/out.dat#PBS -l walltime=10:00#PBS -o hypnos:/tmp/stdout#PBS -e hypnos:/tmp/stderr#PBS -l nodes=2#PBS -vX509_USER_PROXY=/home/abose/.globus/.gass_
27、cache/local/md5/1c/fd/d3/753b9028dfec2ddd6df84cd06c/md5/0a/4b/1d/599dac54863d650c2531cb92fc/data,GLOBUS_LOCATION=/usr/grid,GLOBUS_GRAM_JOB_CONTACT=https:/chi.grid.umich.edu:58963/575/1047861360/,GLOBUS_GRAM_MYJOB_CONTACT=URLx-nexus:/chi.grid.umich.edu:58964/,HOME=/home/abose,LOGNAME=abose,LD_LIBRARY
28、_PATH=#Change to directory requested by usercd /home/abose/usr/gmpi.pgi/bin/mpirun np 4 /home/abose/test.exe t 100 f out.dat Slides taken from NPACI Training, 2003 Command line programs syntax: grid_* or globus_* Function calls/APIs start with globus_* Library binaries start with libglobus_*.a Inclu
29、des:#include /defines most common data structures and others depending on which modules/functions are called in the program. Module Activation/Deactivation:- Functions are arranged in several modules. The corresponding modules must be activated before calling a function:- globus_module_activate(MODU
30、LE_NAME)- globus_module_deactivate(MODULE_NAME)- globus_module_deactivate_all()GLOBUS_SUCCESS (0) is returned if successful. Example Module Names: GLOBUS_GRAM_CLIENT_MODULEGLOBUS_IO_MODULEGLOBUS_GASS_COPY_MODULE Dependencies among module activations exist. Read API documentation. 评 价 优点：增加了对JOB资源的描述
31、定义了很多Attribute，支持GRAM、DUROC等多种资源管理方式 缺点：也是偏重于对计算资源和资源请求的描述，不够广泛可扩展性不好目前仅用于Globus，还不被其他Grid项目所支持WWW服务描述语言WSDLWSDL Web Service Description Language 用于描 述Web服务的技术调用语法。 WSDL定义了一套基于 XML的语法，将Web服务描述为能够进行消息交换的服务访问点的集合，从而满足了这种需求。 WSDL服务定义为分布式系统提供了可机器识别的SDK文档，并且可用于描述自动执行应用程序通信中所涉及的细节。 WSDL的当前版本是1.1，规范可以从http
32、:/www.w3.org/TR/wsdl获得。 WSDL WSDL由Ariba、Intel、IBM和微软等开发商提出。 它用一种和具体语言无关的抽象方式定义了给定Web服务收发的有关操作和消息。 WSDL保持协议中立，但它确实内建了绑定SOAP的支持，从而同SOAP建立了不可分割的联系。WSDL的信息模型 WSDL信息模型充分利用了抽象规范与规范具体实现的分离，也就是分离了服务接口定义(抽象接口)与服务实现定义(具体端点)。 抽象接口规范描述了终端的处理能力，它在WSDL中表示为portType。束定机制 (binding mechanism)在WSDL中表示为binding元素，它使用特定的
33、通信协议、数据编码模型和底层通信协议，将Web服务的抽象定义映射至特定实现。若束定结合了实现的访问地址，抽象端点也就成为可供服务请求者调用的具体端点(concrete endpoint)，WSDL的port元素表示了这一结合。 抽象接口可以支持任何数量的操作(operations)。操作是由一组消息(messages)定义，消息定义了操作的交互定式。与抽象的消息、操作概念相对应的具体实现是由binding元素指定。与XML应用相同，WSDL模式定义了几个高层元素，或称为主要元素。 WSDL描述的基本属性 服务做些什么-服务所提供的操作(方法)。 如何访问服务-数据格式详情以及访问服务操作的必要
34、协议。 服务位于何处-由特定协议决定的网络地址，如URL。 WSDL基本元素的含义元素名含义types定义了Web服务使用的所有数据类型集合，可被元素的各消息部件所引用。 message通信消息数据结构的抽象类型化定义。使用Types所定义的类型来定义整个消息的数据结构。 operation对服务中所支持操作的抽象描述。 portType对于某个访问入口点类型所支持操作的抽象集合。 binding包含了如何将抽象接口的元素(portType)转变为具体表示的细节，具体表示也就是指特定的数据格式和协议的结合 port定义为协议/数据格式绑定与具体Web访问地址组合的单个服务访问点。 servic
35、e定义服务WSDL信息模型 WSDL对象结构图 WSDL文档类型 typestypestypestypesmessagemessageoperationoperationportTypebindingbindingportportportserviceserviceWSDL文档结构WSDL工具 Omniopera-图形用户界面的WSDI、XML和XSD编辑器。 Microsoft的SOAP Toolkit-一种工具包，其中包括根据WSDL定义创建COM接口的向导程序，还包括根据COM接口创建WSDL的向导程序。 IBM的Web Services Toolkit-一种工具包，其中包括产生WSDL
36、和SOAP部署说明的向导程序。 资源描述框架RDFRDF Resource Description Framework, RDF W3C的资源描述框架(RDF)的目的是提供一个访问网络资源元数据(metadata)的标准，因此也提供了一个描述特定资源内容的标准协议。 W3C应用元数据时的推荐标准 是一个模型，一种句法(syntax(es) 应用在Web上时，RDF 通常用XML来编码 是语义万维网 (semantic Web)的基础、支撑W3C - Resource Description Framework (RDF)http:/www.w3.org/RDF/RDF 是一个用于表达关于万维网
37、(World Wide Web)上的资源的信息的语言。 专门用于表达关于Web资源的元数据, 比如Web页面的标题、作者和修改时间，Web文档的版权和许可信息，某个被共享资源的可用计划表等 为什么要使用 RDF? RDF提供共享元数据的模型(model) 共享语义(meaning) 元数据可以在相互了解不多或根本不了解的应用之间共享 例如一个基于RDF的书目应用能够吸收基于RDF的地理空间应用的元数据并对其意义有所理解。用(X)HTML和XML置标后，软件应用必须能够理解复杂的编码RDF的基本思想 用Web标识符（称作统一资源标识符，Uniform Resource Identifiers或U
38、RIs）来标识事物，用简单的属性（property）及属性值来描述资源。这使得RDF可以将一个或多个关于资源的简单陈述表示为一个由结点和弧组成的图（graph），其中的结点和弧代表资源、属性或属性值。 举例有一个人由http:/www.w3.org/People/EM/contact#me 标识, 他的名字是Eric Miller, 他的电子邮件地址是emw3.org,他的头衔是Dr. Eric MillerDr. 统一资源标识符URIURI Uniform Resource Identifiers,URI 是一种简单的可扩展的指定资源的方法URI的同一性 尽管不同资源的访问机制可能不同，但U
39、RI允许不同类型的资源标识符在相同的上下文环境中使用 URI允许用统一的语义解释跨越不同类型资源标识符的通用语法规范 URI可以在不影响已有的标识符系统的情况下，引入新类型标识符 URI允许在多种不同的环境中重用同一个标识符 URI允许新的应用或协议采用已经存在的、广泛使用的资源标识符URI举例 ftp:/ftp.is.co.za/rfc/rfc1808.txt Gopher:/spinaltap.micro.umn.edu/00/Weather/Californian/LosAngeles http:/ mailto: News:comp.infosystem.www.servers.uni
40、x telnet:/159.226.39.252URI=URL+URN URL（Uniform Resource Locators）统一资源定位器 URN（Uniform Resource Name）统一资源名字 从不同角度标识一个资源URL 一般形式是 : scheme: ftp,http,Gopher,mailto,news,nntp,telnet,wais,File,prospero /:/URN :=“urn:”“:” ：名字空间标识符 是符合名字空间规范的字符串 NID :=1,31 :=|“-” := :=| Upper:大写字母，lower:小写字母，number:数字URN :
41、=I* :=|“%” :=|“A”|“B”|“C”| “D”| “E”| “F”| “a”| “b”| “c” |“d” |“e”| “f” := “（”|“）”|“+”| “，”| “-”| “.”| “:”| “=”| “” |“;” |“$”| “_” |“!”|“*”|“” LDAP中的资源描述LDAP 用一系列“属性对”的形式来存储记录项，每一个记录项包括类型和属性值。举例 dn:cn=My Computer, ou=devices, dc=sdu, dc= cn:FB Computer usage:computing resouce:866MHZ resource:512M memory resource:60GB storage resource:Linux OS资源命名资源命名的意义和作用 资源名可以把资源进一步抽象，将资源的标识和资源的位置分离开来 资源命名机制可以建立虚拟空间，扩大或缩小用户空间 实现资源的按名访问，方便用户使用 资源的命名有不同形式 逻辑名称：方便用户，便于记忆 物理名称：实际名字 内部名称：系统内部 外部名称：外部提供用户 命名规则，防止冲突 风格统一 全球唯一