Distributed Virtual Systems Project (DIVIRS)

University of Southern California
Information Sciences Institute

ISI's Distributed Virtual Systems Project is developing mechanisms that enable users to organize, integrate, and utilize the growing number of computing and information resources that are available on large networks, without the need for a central authority to impose structure on the resources.

Commercial online information service providers have been introducing Internet services to allow their users to access information from the World Wide Web, Gopher, WAIS, and Prospero servers on the Internet. The gateway used by America Online and developed by Pandora systems, uses the Prospero Directory Service developed as part of the Distributed Virtual System project to translate data from the Gopher and WAIS formats, returning data to a Prospero client using the Prospero protocol. The Prospero server caches data from other services, reducing network and server load, and improving performance and reliability. The Prospero gateway handles well over 100,000 queries per day, of which approximately 88,000 are handled from the server's cache, reducing unnecessary network queries. Use of Prospero for such gateways was enabled by efforts to optimize performance of the Prospero server. Those improvements are part of the standard Prospero release and therefore benefit all users.

ISI's Distributed Virtual Systems Project is developing mechanisms that enable users to organize, integrate, and utilize the growing number of computing and information resources that are available on large networks, without the need for a central authority to impose structure. Computing resources are managed according to a uniform model that is applicable to both parallel and distributed computing allowing multiprocessor systems to exist in the context of distributed systems, making them more easily shared by those that need them.

Work in this area is based on the Virtual System Model, wherein users construct virtual systems by selecting objects and services that are available over the network. Users then treat the selected resources as a single system, ignoring those resources that were not selected. The resulting architecture provides a middle layer through which network accessible information can be exported by independently developed services, and accessed by different applications.

The Prospero Resource Manager (PRM) has been developed to support the allocation of processing resources in large distributed systems. PRM employs three types of managers: system managers, job managers, and node managers. Each manager controls a subset of the resources in the system, independent of other managers of the same type. The complexity of these management roles is reduced because each is designed to utilize information at an appropriate level of abstraction.

Scalability is improved because independent system managers control resources in different parts of the system. The job manager manages the resources that have been allocated to a job by the system managers responsible for each resource. In so doing, the job manager acts as an agent for the tasks in a job, providing a single entity from which the tasks request resources, and providing the appearance of a virtual system to the application.

TECHNOLOGY TRANSITION

The Prospero Directory Service is available by anonymous FTP to the Internet community. It is in use on more than 50,000 systems in 30 countries on 6 continents. Prospero is used by America Online as the base technology for their Internet information service. Alone, the America Online Prospero gateway handles in excess of 100,000 queries daily, of which about 88,000 are processed from the gateway's cache. Prospero has also been adopted by Bunyip Information Systems as the mechanism for exporting data from archie servers, and new services that they have under development. Among the heaviest uses of Prospero is access to directories of Internet accessible files indexed by "archie" servers scattered across the Internet. The Prospero Directory Service has also been licensed (non-exclusively) by CyberSafe Corporation, which will use it in products it is developing.

Project staff: