This lecture is divided into hyperlinked
sections
This lecture is divided into hyperlinked sections
Introduction
Interactive Multimedia
The Role
of a Multi-media server
Types of multimedia
servers.
How are multi-media
servers used?
The
ways multimedia information is transmitted
Issues
in Real-time Transmission from Multi-media Servers
Modes of Interactive
Access
Video on Demand
Movies on Demand
Performance requirements
News on Demand
Corporate VOD Services
What do
we want from a Multi-media Server?
Scaling
up content/ management and/ or streaming capabilities
Reliability
Storage requirements
Examples
of the use of Multimedia
Conclusion
Resources
In recent years multimedia technology has emerged as a key technology mainly, because of its ability to represent information in disparate forms as a bit-stream. This enables everything from text to video and sound to be stored, processed and delivered in digital form.
We need to understand how multimedia technology deals with this type of information, which is in general task-dependent and is extracted from data in a particular context by exercising knowledge. The desire to deal with information from forms such as video, text and sound will result in a data explosion. This requirement to store, process and manage large data sets leads to the consideration of programmable parallel processing systems as strong candidates in supporting and enabling multimedia technology.
Interactive multi-media (IMM) encompasses interactive marketing, interactive shopping, digital libraries, advertising and advert insertion, digital newsrooms, training-on-demand, video on demand (VOD), near VOD, electronic games, video mail, and more. In definitive terms, IMM is the network delivery of rich digital content, including audio and video, to a client device (e.g. desktop computer, TV and set-top box, network appliance), typically as part of an application having user-controlled interactions.
Thus, a client device, with VCR-like controls, that allows users to reverse, fast forward, pause, and play a video presentation on their screen is one part of IMM. Similarly, a user at a desktop machine scrolling quickly through combinations of documents and video clips, while cutting and pasting various segments together is also using IMM.
The Role of a Multi-media server
Multi-media hardware operating over a network can interact in a symmetrical manner; for instance if we have two multimedia delivery systems, each provided with video camera, microphone and speakers to support video conferencing, they will interact remotely symmetrically since neither has a privileged role.
Strictly symmetrical systems where all resources are equally distributed over a network are however often uneconomic. There has been a trend to pool resources which can be shared onto bigger or specialised systems where economy of scale will be possible.
Resources which can be pooled include disks, software applications such as for database management or CAD and processing power. The pooled resource is on servers, and is accessed by client software
Figure 10.1 Schematic of a Web Broadcasting System
The client server paradigm is particularly applicable to multimedia because of its high processor performance needs and large disk capacity needs.
Image server - a computer provided with large capacity magnetic or optical storage disks holding databases of recorded images; clients are delivery systems which may display images without having to store them on local devices.
Motion video server - similar to image server but with sufficient processing power to output multiple streams of motion video; clients are delivery systems playing back video (or animated) sequences over a network.
TV broadcast server - a gateway between TV broadcasts and the local area network of an organisation; clients will be delivery systems displaying TV broadcasts without TV tuner equipment. The TV video streams are carried by the local area network.
How are multi-media servers used?
Multi-media servers support a variety of different functions. They may act as a repository of information that users access over a network such as video movies or multi-media documents. They may offer remote interactivity pursuing various objectives, cultural or educational, with look-up multi-media documentation, daily life support with online-shopping or ticket reservation, entertainment with video games or video on demand. In all these applications, the user takes the initiative of accessing the information system. These are interactive applications.
In contrast, systems may spontaneously distribute information to users, to all reachable users (broadcast), or only to a selected subset (multicast), and individual users may or may not have some control over the local presentation. This is what we will call distribution applications.
The ways multimedia information is transmitted
There are two ways information can be sent from a server to a client:
Asynchronous transmission (download, store and present) Here the information, or part of it, is first transmitted, then stored at the receiving end, and further displayed. The information may be either fetched by the user or spontaneously distributed by the server.
Synchronous transmission (Real-time transmission) Here part or all of the information is transferred in real time over the network for on-the-fly presentation on the receiving system.
With text information, either synchronous or asynchronous transmission can be used; there are no great demands in terms of storage requirements or on the processing power of server, client or network. However, if large volumes of multi-media information are involved such as video sequences, it is unlikely that the entire sequence can held in the store of a client system, so that asynchronous transmission would be reserved for relatively small sequences that can readily be stored, perhaps in the client’s hard disk.
The playback will take place immediately after the downloading is complete. Long sequences require real-time transmission which creates much more stringent demands on both the underlying network and the client system. Most systems designed for remote access of multi-media documentation do not inherently support real-time transmission of continuous media. As a result the contents of the multi-media document must be structured in such a way that long sequences are cut into individual pieces. Unless considerable storage capacity exists in the client, real- time transmission is the only option.
Issues in Real-time Transmission from Multi-media Servers
Temporal relationships exist between the various parts of the multi-media information being transferred to a client; that is graphics, sound and motion pictures and text must be synchronised. When transferred across a network, these temporal relationships may be altered because of delays (latency) introduced by the network, so that certain fragments of the various streams of data may be delayed more or less than others. If such timing alterations exist, mechanisms must be set up jointly in the server and client system to:
1. Restore the temporal relationships within each stream carrying the continuous media information. This is called streaming. Streaming means ensuring the received audio, motion video, and animation streams are smoothly re-arranged to reassemble the original stream.
2. Restore the temporal relationships between the various streams or elements which compose the multi-media information. This is called synchronisation.
Users may want to access a multi-media server with either of the following objectives:
Retrieval only - The objective here is to locate, display and possibly record multi-media information. This is the look-up of a multi-media document, the display of a movie, or learning with a multi-media self-learning guide. A schematic of this process can be seen below.
Figure 10.2 A user requests Interactive material
Retrieval with processing of user input - The goal
here is not solely access to information, rather access is only a means
to another objective; buying items, booking tickets, scoring educational
exercises. Examples are online-shopping where the display of multi-media
is the means by which users may be persuaded to buy. Another example is
active online-education via multi-media programs where exercises are proposed
and responses corrected and recorded. Such applications are called transaction-orientated
applications.
Figure 10.3 Operation of Transaction Oriented Applications
VOD covers a range of applications where users request access to video servers for still or moving pictures on an individual basis. Mostly, but not exclusively, the term is applied to access to motion video. VOD may be a corporate service provided on site within an organisation via a local area network; alternatively it may be a public service.
True VOD systems are extremely demanding in terms of power to access and read the data storage devices, processing power of the server hardware, and bandwidth of the network. For instance requests for a popular movie in peak hours may number 1000s. It is likely that all requests for the movie cannot be satisfied immediately after the request for all users. In circumstances such as this the server will supply the movie for the users one after the other, so that some users will experience a delay between making a request and subsequently having the movie downloaded.
Compared to other types of data files, digitised video files are enormous. Digitised movies of the two-hour variety, for example, in highly compressed form, still require a billion or more bytes of data each. Thus, a large-scale system, serving users with thousands of different content products would require thousands of billions of bytes of storage (i.e. terabytes of storage). In non-compressed form, these files are impractical to store, and impossible to transfer over the largest bandwidth networks currently available without overwhelming them. Therefore, every multi-media solution assumes compressed-file storage and transfer with on-the-fly decompression at the desktop system or set-top-box equipped television. The MPEG-2 compression/ decompression standard is the choice for digital movie file storage and transfer. Motion JPEG standards are used in so-called ‘digital newsroom’ applications because this standard permits motion-image-file editing.
This is the VOD service supplied to the public or to clients in a hotel where the objective is to access a stored movie. The objectives are to give a wide range of choice in movies and/ or to offer an alternative to a conventional VCR, eliminating fetch/ return and manipulation of physical video tapes. Ideally all VCR functions should be available during the movie's delivery, such a fast forward, rewind etc. This is fully interactive VOD. In practice early systems either offered no degree of interaction, or only allowed a limited set of control functions, pause for instance.
A VCR has very limited interactivity. The content is stored on tape, and the end user is the viewer equipped with a television set and remote control. Only one signal stream is being sent over one signal path. When you want to pause the signal, the pause button on the remote control is pressed, and the VCR responds well within a second.
Now consider a system consisting of 100 VCRs loaded with 100 different videos. The outputs of those VCRs are connected to a switching matrix which is also connected to 500 television sets. Since the television sets and VCRs are not in the same room, remote control signals are sent via the wires interconnecting the television sets and VCRs. Now, to make this VCR system work, a designer must build a network system which determines from the remote-control signal the correct VCR to put in play mode and to which television to route that output.
The device must be capable of responding to up to 500 remote control signals at any time and doing so with delays commensurate with the single VCR/ television set up.
Take this analogy and enlarge it 100-fold and make it more complicated by adding a variety of output delivery paths and some variations of on-line and off-line storage subsystems. This magnified view offers a glimpse of the complexity required to match content with end user, while giving the end user interactive control. Keep in mind that this VCR system analogy is the tip of an iceberg relative to a media server’s requirements and functionality. It is limited in content, limited to video, offers much less interactivity, and would be much more expensive to implement on a scale comparable to a modest media server. But this example does serve to give one a feel for the kinds of capabilities and functions a media server would provide.
A picture quality equivalent to a VCR cassette needs a bit rate of 1 Mbps. however it seems that a quality lower than that of NTSC (National Television Systems Committee) may not be readily accepted by potential customers. This is the standard in use in the USA and Japan (525 lines per frame and 30 frames per second) or PAL/ SECAM (PAL - Phase Alternating Line 625 lines and 25 frames per second) is the name of the broadcast colour television standard used in Germany, UK, South Africa, parts of Asia and other parts of Africa.
Early implementations of the MPEG-2 (Motion Picture Expert Group) standard operate at 6 Mbps with a quality somewhat better than broadcast TV. MPEG refers to a group of standards for coding and compression of digital motion video and associated audio. MPEG-2 is the emerging standard which targets studio quality television and multiple CD quality audio at bit rates of from 4 to 6 Mbps. MPEG-2 has also been extended to cover HDTV (High Definition TV).
Various compression techniques may be used for digital quality TV but the MPEG-2 standard has emerged as a general multi-vendor technology. When using MPEG-2, operators may also take advantage of the rich palette offered by the associated audio schemes. For example, MPEG-2 audio has foreseen 5 channels which could be used to transmit the soundtrack in the user's language of choice with CD quality stereo sound only requiring 192 or 256 Kbps.
HDTV movie-on-demand is the next challenge. When using MPEG-2 compression, the required bit rate will range from 15 to 34 Mbps. depending on the actual resolution and frame rate required.
As well as movies other types of stored information may be offered such as news, and give delivery of multi-media news including textual and still picture information. offering a higher degree of interaction than is usual from conventional movie-on-demand services; this is so-called public news-on-demand. The presentation device is either a personal computer or more frequently a conventional TV connected to a separate set top box.
Publishers of business and financial news are becoming increasingly involved in supplying news-on-demand services, targeting professional subscribers. This is called business news-on-demand. As with public news-on-demand, the tendency is to offer business news in multi-media form and to mix the ‘on-demand’ mode with up-to-the -minute information such as financial news reports. The presentation device is generally a personal computer or workstation.
The differences between corporate and public VOD are:
Firstly, private servers need to store far fewer video sequences than public servers; nothing like the 1000 or more typical of a public VOD service.
Secondly the number of subscribers is far fewer. It is not untypical for a public VOD service to have several hundred thousand subscribers.
Thirdly on-site corporate networks give much higher bit rates than available over wide area systems.
The net result is that corporate VOD servers are simpler and can be less powerful than their public counterparts. Often a high end workstation will be adequate, provided there is provision for handling video files and with suitable disk subsystems, possibly some RAID systems.
Applications include individual training and education with movies delivered directly from server to desktop client machines, access to corporate video news and access to stored corporate seminars or presentations.
What do we want from a Multi-media Server?
The ideal media server must be able to expand to accommodate increased streaming, or increased content storage and management. Streaming depends upon four resources:
· I/ O for gathering content from storage,
· network output for delivering digitised content
to the network,
· interconnect bandwidth for connecting the content
I/ O to network output,
· CPU resources to orchestrate the process.
These four resources have to scale in proportion to one another. If not, money is being spent on excess and unutilised resources.
Scaling of content storage and management is critical for associating information products with various services and applications. Content has to be available, via the network, to all potential end users. As content grows, so must I/ O bandwidth, and interconnection bandwidth and connections. Balance is achieved by mixing the right proportions of I/ O bandwidth and interconnection bandwidth with CPU resources for managing this process.
A modular approach to media server design that was partitioned into streaming oriented and content/ management-oriented modules would allow the system to scale in a balanced way even as proportions of scaling and content changed with time.
Scaling up content/ management and/ or streaming capabilities
By designing the modules appropriately, one type of module can be specific for scaling up content and management; the other can be used to scale up content streaming. Both modules would also provide incremental increases in CPU and interconnection functionality. This would permit one to tailor a system to current requirements, then to easily expand its capabilities in content storage, content streaming, or both. In any case, adding one or both types of modules can increase the media server's capabilities while preserving the right balance of resources.
Reliability is critical for IMM. Whether public network and entertainment applications, or private network and business applications, an IMM system has to be highly available. This means, it has to be highly reliable and rapidly repairable. The lessons of other segments in the electronics industry are that fewer components add up to increased mean times between failures (MTBF).
Modules that can be "hot swapped", that is, pulled out
or inserted without having to power down or interrupt the system, shorten
the mean times to repair a system. An ideal media server would be highly
integrated (i.e. fewer components) and offer
hot-swap module capabilities.
The cost-per-stream of media server hardware can be calculated
by dividing the hardware cost by the maximum number of video streams supported.
It should come as no surprise that mainframe computers have a relatively
high cost per stream. It should also come as no surprise that for very
limited applications, media severs based on
off-the-shelf microprocessor technology offer a low cost
per stream. But a better metric is to project the cost per stream over
a reasonable scale of expansion. When that is done, one finds that the
cost per stream of the mainframe goes up as the scale of the implementation
goes down. The cost per stream for systems based on off-the-shelf microprocessors
goes up rapidly, as the scale of the implementation goes up.
Since a media server is a combination of hardware and software, and the software will be offered by different vendors, the media-server-hardware vendor must provide an open architecture to which effective media-server software can be ported. This can be done by providing a set of open services to which the media-server software can be ported.
As for the network end of things, ATM (Asynchronous Transfer Mode) is perhaps the transport technology of choice for today's and tomorrow's multimedia-capable networks. Thus, ideal media server hardware would provide an ATM network interface. This would accommodate ATM-ready LANs and WANs. It would also allow transporting MPEG content over ATM by virtue of the standards being developed by the ATM Forum.
What kind of storage best meets the need of IMM? In fact, there is no best type of storage. It is reasonable to expect that multi-media content will be stored in various combinations of solid-state memory, magnetic disk storage, optical storage, and magnetic tape storage. In the order shown, these technologies represent storage options that decline rapidly both in cost and transfer speed
The line of acceptability is MPEG-1 at a sustained transfer rate of 1.544 Mbps (equal to a T1 channel rate) in order to support VCR video quality. Higher image quality, using schemes like MPEG-2, will push the rates up to 6 Mbps, and even as high as 50 Mbps for some applications.
Disk drives, optical drives, and solid-state memories can deliver that T1 speed; tape storage systems cannot. Thus, while tape represents a low-cost, high-capacity means for storing multi-media content, it must first transfer its contents to either disk or RAM before the content an be streamed at an acceptable data transfer rate.
The types and organisation of the various content-storage options will depend upon several factors, including the maximum number of digital streams, the maximum permissible response latencies, and the distribution in frequency of access for the different content products. In other words, if a system is meant to deliver a few dozen extremely popular movies, most of the storage will be of the fast, on-line type. On the other hand, if a system will manage hundreds or thousands of titles that vary greatly in how frequently they are accessed, then one would expect a greater proportion of archival (e.g. tape) storage than in the first system.
Examples of the use of Multimedia
A telemedicine project teams NPAC and the State University of New York Health Science Centre in a project to deploy collaboration technology over high-speed networks. Two and three dimensional image processing will be integrated with collaboratory software, to link university hospital specialists with general practitioners in rural hospitals. This will strengthen pathology, surgery, and trauma care in a large health care delivery region.
A new initiative between NPAC and the Newhouse School of Public Communications at Syracuse University will prototype digital wire service delivery between news broadcasters and their affiliates. Live satellite feeds of video source material will be captured, digitised, compressed, and stored in real time, with off line indexing. A multi-resolution format for video and image content provides low resolution browsable material, as well as high resolution broadcast quality material.
Interactive multi-media is an important source of information on demand, from movies to educational resources. This allows moving images and audio to be selected by a user and delivered to him/ he via a network.
The resources are held on powerful servers with large hard disks and high performance processing capabilities, connected to a network via a high performance I/O connection. The network itself needs to be optimised to deliver the resources in a timely manner. One technology that can support this is ATM over optical fibre (SONET/ SDH).
Resources may be broadcast to many clients simultaneously, to a selected group of clients (multicast) or delivered to single clients (unicast).
Clients should have the capability to pause, fast forward or rewind the resource that has been delivered.
Resources may be transmitted and stored before the client displays them (asynchronous transmission) or broadcast in real-time (synchronous transmission). The method used will depend on the storage capability of the client and the volume of data that the resource consists of.
Multimedia may be accessed on a retrieval only basis where the data is delivered directly to the user according to his/ her request, or may be part of a larger scheme where the multimedia is part of an interactive session with the client e.g. a training video that has questions for the user to respond to. The user’s responses are recorded by the server. This is an example of a transaction oriented application.
Video on demand is currently available on a pay-per-view basis from certain satellite broadcasting companies. Users are able to choose a film to view and it is ordered using the user’s telephone connection. The films are set to begin at 10 or 15 minute intervals and are broadcast on separate channels.
The films are compressed using MPEG-2 technology which is able to deliver high definition video with stereo sound on a T1 connection which can deliver 1.544 Mbps. Without compression, the storage and delivery of films by digital means would be impossible due to the bandwidth required. MPEG technology is also used to compress music for delivery by the Internet, an example of a service on offer is Napster.
News may also be delivered to a set of users on demand, corporate VOD can be implemented to allow employees to view work-related resources. The benefit here is that the server need not be as complex as true VOD systems as the company will have a high speed LAN and the amount of stored material will be lower than that of a satellite broadcaster.
IMM hardware must be reliable and fast to repair. Typical hardware will have hot swappable hard disks and possibly employ RAID technology. This will reduce the MTBF.
ATM is a technology that has been designed to carry high bandwidth content material across arbitrary networks with little latency and low jitter. It is already implemented in many WANs but is not often deployed in LANs.
The multimedia material can be stored on a variety of
media, the cheapest being tape, however the tape is unable to be used for
direct broadcasting as the data cannot be read from the tape at the rate
required for transmission onto the network. Magnetic disk and optical storage
media are the choices for direct broadcasting of IMM.
An excellent web based VOD resource
http://lal.cs.byu.edu/ketav/issue_2.5/vod/vod.html