Grid Computing Part 6. Information Service Margarita M.Stepanova mstep@mms.nw.ru Saint-Petersburg State University, Faculty of Physics v.1.5 27.03.11. - презентация

1 Grid Computing Part 6. Information Service Margarita M.Stepanova Saint-Petersburg State University, Faculty of Physics v

2 Owerview The Information Service (IS) provides information (static and dynamic ) about the Grid resources and their status. A basis for configuration and adaptation in heterogeneous, dynamic environments Requirements and characteristics –Uniform, flexible access to information –Scalable, efficient access to dynamic data –Access to multiple information sources –Decentralized maintenance

3 Globus Monitoring and Discovery Service (MDS2) MDS provides access to static and dynamic information of resources. It contains the following components: Grid Resource Information Service (GRIS) Grid Index Information Service (GIIS) Information Provider (IP) MDS client Based on the Lightweight Directory Access Protocol (LDAP)

4 MDS as Grid service

5 The interconnection of the MDS components

6 Абстрактная модель, отражающая объекты реального мира в конструкции, которые можно реализовать в компьютерных системах Определяет какая информация и каким образом будет храниться в MDS GLUE Schema Grid Laboratory Uniform Environment Общая информационная модель для Грид Определяет данные и их структуру в MDS На данный момент используется схема GLUE-1.3 Реализована в виде LDAP и XML схем Вышел и внедряется новый стандарт - схема GLUE-2.0 LDAP/XML информационные модели основаны на entries (объекты типа host, person, service, etc.), каждый из которых имеет одно и несколько свойств или атрибутов. Каждая entry имеет уникальный идентификатор Distinguished Name (DN) и каждый атрибут имеет свой тип данных и значение. Information model (GLUE Schema based)

7 GLUE-1.3 Schema: Site

8 GLUE-1.3 Schema: Computing Element

9 Information providers - translate the properties and status of local resources to the format defined in the schema and configuration files. Grid Resource Information Service (GRIS) - GRIS is the repository of local resource information derived from information providers. -The local information maintained by GRIS, is updated when requested, and cached for a period of time (TTL) - GRIS is able to register its information with a GIIS, but GRIS itself does not receive registration requests - Server which runs on each resource (port = 2135)

10 Grid Index Information Service (GIIS) -GIIS is the repository that contains indexes of resource information registered by the GRIS and other GIISs. -GIIS has a hierarchical mechanism, like DNS - port 2170 */ gLite: - the GIIS uses a Berkeley Database Information Index (BDII) to store data, which is more stable than the original Globus GIIS - BDII is also used as the top level of the hierarchy - The top-level BDII obtains information about the sites in the Grid from the Grid Operations Centre (GOC) database

11 Example: gLite 3.0 Two IS systems used: Globus Monitoring and Discovery Service (MDS) -> BDII - used for resource discovery and to publish the resource status Relational Grid Monitoring Architecture (R-GMA) - used for accounting, monitoring and publication of user-level information

12 gLite Information System (BDII) The resource level BDII is usually co-located with the grid service and provides information about that service. Each grid site runs a site level BDII. This aggregates the information from all the resource level running at that site. The top level BDII aggregates all the information from all the site level an hence contains information about all grid services. The information system clients query a top level BDII to find the information that they require. Hierarchical structure of three levels:

13 The information in the information systems conforms to a schema called the GLUE schema. The Berkeley Database Information Index (BDII) consists of a standard LDAP database which is updated by an external process. The update process obtains LDIF from a number of sources and merges them. It then compares this to the contents of the database and creates an LDIF file of the differences. This is then used to update the database BDII: update process

14 MDS Commands APIs for accessing & updating MDS contents –C, Java, PERL (LDAP API, JNDI) Various tools for manipulating MDS contents –Command line tools, Shell scripts & GUIs LDAP defines a set of standard commands ldapsearch, etc. We also define MDS-specific commands –grid-info-search, grid-info-host-search

15 Example: Filtering % grid-info-host-search -L (objectclass=GlobusSoftware) dn: sw=Globus, hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl, dc=gov, o=Grid objectclass: GlobusSoftware releasedate: 2000/04/11 19:48:29 releasemajor: 1 releaseminor: 1 releasepatch: 3 releasebeta: 11 lastupdate: Sun Apr 30 19:28:19 GMT 2000 objectname: sw=Globus, hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl, dc=gov, o=Grid

16 Example: Attribute Selection % grid-info-host- search -L (objectclass=*) dn hn –Returns the distinguished name (dn) and hostname (hn) of all objects –Objects without hn fields are still listed –DNs are always listed dn: sw=Globus, hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl, dc=gov, o=Grid dn: hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl, dc=gov, o=Grid hn: pitcairn.mcs.anl.gov dn: service=jobmanager, hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl, dc=gov, o=Grid hn: pitcairn.mcs.anl.gov dn: queue=default, service=jobmanager, hn=pitcairn.mcs.anl.gov, dc=mcs, dc=anl,dc=gov, o=Grid

17 LDAP directory of an gLite 3.2 BDII

18 Grid Monitoring Architecture (GMA) From GGF Very simple model Does not define: –Data model –Data transfer mechanism –Registry implementation Producer Consumer Registry Store location Lookup location execute or stream Defined in GGF Document: GFD-I.7 Use the GMA from GGF A relational implementation –Powerful data model and query language All data modelled as tables SQL can express most queries in one expression Applied to both information and monitoring Relational Grid Monitoring Architecture (R-GMA):

19 gLite Information System: R-GMA. The R-GMA (Relational Grid Monitoring Architecture) servlet accepts connection from clients (producers), i.e. the services publishing and user jobs publishing information, and forwards the information to the appropriate consumers. R-GMA is composed of the following services: R-GMA server R-GMA client is a set of client API in C,C++,Java and Python for the access the information and monitoring functionality of the R-GMA system R-GMA site-publisher (each site) is responsible for publishing site information to the R-GMA server. R-GMA service tool -regularly scans config files and updates a current Service Status R-GMA Table. CLI exists for modify these files and query the service table.

20 R-GMA Browser

21 GT 4.0: Information Services (MDS 4 or WSMDS) Aggregator Services – сервисы сбора информации –Index Service - сервис индексации, сбор информации и представление ее в виде свойств ресурса вышестоящим сервисам –Trigger Service – сбор информации, сравнение с шаблоном и выполнение действий при совпадении. Information Interfaces and Providers – интерфейсы и сервисы предоставления данных –Aggregator Framework – база для разработки сервисов сбора и предоставления информации о Грид –Information Providers – провайдеры информации о ресурсах и сервисах –WebMDS user interface – веб-интерфейс для предоставления информации поставляемой согласно спецификациям Web Services Resource Framework (WSRF) The full WSRF implementation of MDS, which consists of

22 Aggregator Framework architecture: Information Flow in Web-MDS

23 MDS4 and preview version MDS a more powerful query language (Xpath instead of LDAP); a simpler and therefore more reliable implementation; simpler (indeed, in simple cases, totally automated) configuration convenient interface to arbitrary information sources due to extensible architecture; lower performance due to use of XML protocols

24 Реализация ИС в ГридННС Информационная система: сбор данных агрегирование выдача данных Основное назначение ИС: обслуживание Менеджера заданий Основа: WS-MDS из GT4 Структура данных: GLUE-1.3 Формат хранения и представления данных: XML

25 References