Monday, March 27, 2006 

How is Multimedia Data different

  1. The content of multimedia data is often captured with different “capture” techniques (e.g., image processing) that may be rather unreliable. Multimedia processing techniques need to be able to handle different ways of content capture including automated ways and/or manual methods.

  2. Queries posed by the user in multimedia databases often cannot come back with a textual answer. Rather, the answer to a query may be a complex multimedia presentation that the user can browse at his/her leisure. Our framework shows how queries to multimedia databases may be used to generate multimedia presentations that satisfy users queries-a factor that is unique to our framework.

  3. Multimedia data is large and affects the storage , retrieval and transmission of multimedia data.

  4. In case of video and audio databases time to retrieve information may be critical ex(Video on demand).

  5. Automatic feature extraction and Indexing: In conventional databases user explicitly submits the attribute values of objects inserted into the database. In contrast, advanced tools such as image processing and pattern recognition tools for images, to extract the various features and content of multimedia objects. As size of data is very large we need special data structures for storing and indexing.

delicious icon   digg icon   technorati icon   furl icon   |   2:41 AM | |

Sunday, March 26, 2006 

Types of Multimedia Data

There are number of data types that can be characterized as multimedia data types. These are typically the elements or the building blocks of ore generalized multimedia environments, platforms, or integrating tools. The basic types can be described as follows :

  1. Text : The form in which the text can be stored can vary greatly. In addition to ASCII based files, text is typically stored in processor files, spreadsheets, databases and annotations on more general multimedia objects. With availability and proliferation of GUIs, text fonts the job of storing text is becoming complex allowing special effects(color, shades..).

  2. Images : There is great variance in the quality and size of storage for still images. Digitalized images are sequence of pixels that represents a region in the user's graphical display. The space overhead for still images varies on the basis of resolution, size, complexity, and compression scheme used to store image. The popular image formats are jpg, png, bmp, tiff.

  3. Audio : An increasingly popular datatype being integrated in most of applications is Audio. Its quite space intensive. One minute of sound can take up to 2-3 Mbs of space. Several techniques are used to compress it in suitable format.

  4. Video : One on the most space consuming multimedia data type is digitalized video. The digitalized videos are stored as sequence of frames. Depending upon its resolution and size a single frame can consume upto 1 MB. Also to have realistic video playback, the transmission, compression, and decompression of digitalized require continuous transfer rate.

  5. Graphic Objects: These consists of special data structures used to define 2D & 3D shapes through which we can define multimedia objects. These includes various formats used by image, video editing applications.

delicious icon   digg icon   technorati icon   furl icon   |   5:22 PM | |

Monday, March 13, 2006 

Multimedia Indexing

The Basic Steps for multimedia indexing is :
  1. Provide a measure for the distance between two multimedia objects :
    So we need to define a distance function D().Given two objects OA, OB the distance (=dis-similarity) of the two objects is denoted by D(OA, OB). It can be Euclidean Distance .
  2. Similarity Queries :
    • Whole query matches : for a query of object O , find the objects within the database which are with in similarity distance delta from O.
    • Sub-pattern Match : Given a collection of N objects O1,…, ON and a query (sub-) object Q and a tolerance delta identify the parts of the data objects that match the query Q .
    • Or we can find k nearest neigbours of object queried.
  3. Algorithm : Sequential scanning and distance calculation with each and every object is too slow for large databases. We can use highly fine-tuned database SAMs (Spatial Access Methods) like R-trees to accelerate the search (by pruning out large portions of the database that are not promising) .
The similarity distance can be based on features extracted from image. Like in previous post for an image the similarity can be based on color, texture etc

delicious icon   digg icon   technorati icon   furl icon   |   12:39 PM | |

Wednesday, March 08, 2006 

Case Studies

MediaDB is Multimedia Database Management System that creates MMDB applications to manage large volumes of multimedia objects. MediaDB is an object oriented system that supports encapsulations, inheritance hierarchies, and object management. It also supports one-to-many and many-tomany relationship between objects.

InterBase is a relational Database that has builtin support for BLOBs. It has its own interface to relational database that it manages. InterBase stores BLOBs in collection of segments. A segment in InterBase can be thought of as a fixed-length "page" or I/O block. InterBase provides special API calls to retrieve and modify segments. It allows user to open-BLOb , create-BLOB, read segmet (by get-segment) and write a segment to BLOB (put-segment).

Sybase's SQL Server supports TEXT and IMAGE data types. TEXT and IMAGE data-types can be very large up to 2GB. User can defien table as :
CREATE TABLE Page (
Number Integer ,
Content IMAGE ,
Anstract TEXT)
In terms of their internal storage representation TEXT and IMAGE column values contain pointer to firest page of multimedia column. The pages are stored as linked-list. The pages od TEXT and IMAGES colums are stored seperately fro tables of database.

XDP from Plexus is imaging database engine that provide support for BYTES and TEXT data types. A special family can be assigned to each IMAGE and TEXT column in a Table with a different storae extent or family for that tabel recors. Example :
CREATE TABEL Pages
(Number Intege,
PageImage IMAGE IN ImageFamily)
IN CompoundDocumentFamily

XDP supports hierarchial storage system, managing magnet disks, optical disks, optical jukeboxes with online, near-line (stored in the optical jukeboxes but not munted), and offline stored on a shelf) devices.

MultiMedia Datatypes in UniSQL/X Being an object relationa; database management system , UNiSQL supports most of SQL constructs. FOr multimedai data types UniSqL/X supports a class hierarchiay rooted at generalized large object (GLO) class. This class serves as the root od multimedia data types and provides a number of builot-in attributes and methods for instatiating and inheriting clients.For content of GLO objetcs user can create either Large Object (LO) or a File Based Object(FBO) . An interestin gfeature is that recovery is supported for FBOs but consisteny is not supported as external application can aloso access it. LOs are similar to BLOBs in relational databases. FBOs are stored in host file system. Two other subclasses are Audio and Image class. Audio serve as root class of various type of audio classes. Image class is the root of image types, including 2D and 3D images. It supports number of attributes and ethods(also inherited from parent classes) to support storing and manipulating data.

delicious icon   digg icon   technorati icon   furl icon   |   5:34 PM | |

Saturday, March 04, 2006 

Content Based Retrival

Content-based retrieval (CBR)of a multimedia data is retrieval that uses the text of the document rather than any added metadata. Free text searching is a good example of content-based text retrieval for text documents. The words making up the content of the document are indexed and used as the basis for retrieval, sometimes in conjunction with quite sophisticated “intelligent” software used to satisfy the query. But in case of images and videos CBR is difficult. The difficulty is that an image is an collection of pixels with color specified for each pixels but need to interpret them automatically and see meaningful regions of colour, recognise objects and identify scenes which can usefully form the basis of effective image matching processes.
How can CBR be performed for images :
  • Image Retrieval By Color : The colour histogram has been a simple and popular representation which captures the relative amounts of each colour in an image. But it is a global measure and does not give information about colour variations at local positions in the image. Color Histogram gives the proportio of pixels of each color within a image. Now we define a distance color set distance d(set)=(cq-ct)tA(cq-ct) where
    • cq: Query color histogram
    • ct: Target color histogram
    • A is the color similarity matrix.
    Now for a query we search for minimum value of this distance.
  • Region Query for Images by Spatial description : Image may be divided into some regions. Here we can perform triangulation of image. Now region location can be found using Euclidean distance of centroids. When we query for an region of an image we can make the weighted sum of
    • Color set
    • Location
    • Area
    • Spatial extent
    as a criterion. We may also need to define spatial relation between regions for example
    • Adjacency
    • Nearness
    • Overlap
    • Surround
How can we perform CBR :
1. The application provides us a drawing tool so that we can draw image and calculate feature of query image and search accordingly in database.
2. Or we can prform Query By Example : in this case we provide an example image so that an iomage from database should be similar to it.

delicious icon   digg icon   technorati icon   furl icon   |   9:18 AM | |

Friday, March 03, 2006 

Using Metadata for managing Multimedia Data

As in case of multimedia data the exact match paradigm for querying is no longer suitable or adequate. Also content-based processing are hard to analyse and result is not often precise. So Metadata that is derived or extracted from digital media have advantage of being more amenable to traditional data retrieval and manipulation techniques than raw digital media.

Metadata can often be stored in relatively standard relational and object oriented structures and can be easily queried though simple extension of languages such as SQL.

Type of Metadata :
  • Content Independent Metadata : this type of metadata captures information that does not depend on content of document with which it ios associated. Example of this type of metadata are : owner,location, modification date of document. There is no information content captured but still such metadata is useful for retrival of documents.
  • Content dependent Metadata : Example is size of document,number of rows and columns and othre field which depend upon content. This can be further sub-divided in two :
    • Direct Content Base Metadata : This is based directly on content of document. A popular example is "index for a document" .
    • Content Descriptive Metadata : This type of metadata describes the content of document without direct utilization of those contents.Example is "texual anonations describing an image".
Now here is how can we formally define metadata :

A media-instance is an 8-tuple
(ST, fe, ATR, A, R, F, Var1, Var2)
where:
ST is a set of objects called states, and
fe is a set of objects called features, and
ATR is a set of objects called attribute values, and
A is a map from states to sets of features, and
R is a set of relations on fe^i * ST for i > 0, and
F is a set of relations on ST, and
Var1, is a set of objects, called variables, ranging over ST, and
Var2, is a set of variables ranging over fe.

A state is smallest chunk of media data that we wish to consider. For example , in an image database each eimage may be viewed as state.
A feature is any object in a state that is of interest for multimedia database. Example for an image pic.gif showing car, the feature of pic.gif is car .
The Attributes are properties of features in a state. like for picture of car the attributes may be car color, model.
A is a feature extraction ap which tells which feature occur in which states.
R is set of relation on fe X ATR X ST --those relations are state dependent. for instance for an omage photo.gif. There may be state dependent relation ship called left_of containg tuples like (a , b , photo.gif) such that a is in left of b in photo.gif.
F is a relation on states like "before" example for such relation is (a.gif , b.gif) such that acc to before relation , photo a.gif was taken before b.gif.

Having this formal representation now we can easily formulate methods for queying database easily. Now we cab define functions like : FindObjectWithFeature(a) , this wll search database for all such states with feature containg a. Or we can list all features of an image with FindFeatures() . Other function types can be FindObjectWithFeatureandAttr(Jane,suit,blue) this query asks to find all media objects in which jane apperars in a blue suit. the above mentioned functions are available by default in SMDS-SQL. An example query is "find all images containg jane and denis , with jane to left of Denis.
SELECT M
FROM smds source1 M
WHERE FindType(M) = Image AND
M in FindObjWithFeature(Denis) AND
M in FindObjWithFeature(jane ) AND
left(jane,denis,M)

delicious icon   digg icon   technorati icon   furl icon   |   4:51 AM | |

 

Information Management & Retrival : Introduction

Basic Approches for Information management :

  • Conventional database system – This is the widely-used approach to manage and search for structured data. All data in a database system must conform to some predefined structures and constraints (i.e., schemas). To formulate a database query the user must specify which data objects are to be retrieved, the database tables from which they are to be extracted and predicate on which the retrieval is based. A query language for the database will generally be of the artificial kind, one with restricted syntax and vocabulary, such as SQL.
  • Information retrieval (IR) system – IR system is mainly used to search large text collections, in which the content of the (text) data is described by an indexer using keywords or a textual abstract, and keywords or natural language is used to express query demands. For example for an image or video we have to describe it in words or in a way need to store lot of metadata (texual form).
  • Content based retrieval (CBR) system – This approach is used to retrieve desired multimedia objects from a large collection on the basis of features (such as colour, texture and shape, etc.) that can be automatically extracted from the objects themselves. Although keyword can be treated as a “feature” for text data, traditional information
    retrieval has much more higher performance than content-based retrieval because keyword has the proven ability to represent semantics, while no features have shown convincing semantic describing ability. But major drawback of this method is that it lacks precision.
  • Graph or tree pattern matching – This approach aims to retrieve object sub-graphs from an object graph according to some denoted patterns.

delicious icon   digg icon   technorati icon   furl icon   |   2:59 AM | |

Wednesday, March 01, 2006 

Advanced data structures

k-dimensionl trees (k-d trees) :
Each level of a k-d tree partitions the space into two.
  • choose one dimension for partitioning at the root level of the tree.
  • choose another dimensions for partitioning in nodes at the next level and so on, cycling through the dimensions.
In each node, approximately half of the points stored in the sub-tree fall on one side and half on the other. Partitioning stops when a node has less than a given maximum number of points.

Each line in the figure (other than the outside box) corresponds to a node in the k-d tree. The maximum number of points in a leaf node has been set to 1.






Division of Space by Quadtrees :

Each node of a quadtree is associated with a rectangular region of space; the top node is associated with the entire target space.

Each non-leaf nodes divides its region into four equal sized quadrants correspondingly each such node has four child nodes corresponding to the four quadrants and so on. Leaf nodes have between zero and some fixed maximum number of points (set to 1 in example).




R trees :
R-trees are a N-dimensional extension of B+-trees, useful for indexing sets of rectangles and other polygons. Supported in many modern database systems, along with variants like R+ -trees and R*-trees.
A rectangular bounding box is associated with each tree node.
  • Bounding box of a leaf node is a minimum sized rectangle that contains all the rectangles/polygons associated with the leaf node.
  • The bounding box associated with a non-leaf node contains the bounding box associated with all its children.
  • Bounding box of a node serves as its key in its parent node (if any)
  • Bounding boxes of children of a node are allowed to overlap.
A polygon is stored only in one node, and the bounding box of the node must contain the polygon
The storage efficiency or R-trees is better than that of k-d trees or quadtrees since a polygon is stored only once.
To find data items (rectangles/polygons) intersecting (overlaps) a given query point/region, do the following, starting from the root node:
  • If the node is a leaf node, output the data items whose keys intersect the given query point/region.
  • Else, for each child of the current node whose bounding box overlaps the query point/region, recursively search the child.

delicious icon   digg icon   technorati icon   furl icon   |   3:33 PM | |

 

SQL and Multimedia : Introduction

Here is example on how to define table with large objects i.e LOB

CREATE TABLE grape
( grape_name VARCHAR2(30) ,
grape_txt CLOB DEFAULT EMPTY_CLOB() ,
picture BLOB DEFAULT EMPTY_BLOB() ,
CONSTRAINT prim_grape PRIMARY KEY (grape_name) )

here grape_text is the the description of grape and is stream of characters so is stored as CLOB. Whereas picture is stored as BLOB. Here the image will be stored in database itself. Other way is we can store it as BFILE. Then we can have used :
picture BFILE ,

Now while inserting data we have to do as :
INSERT INTO grape (grape_name, picture) VALUES ('chardonnay' , BFILENAME('PHOTO_DIR','chardonnay.jpg'))

Using DBMS_LOB Package :
CREATE PROCEDURE wine_read_bfile
IS
Lob_loc BFILE ;
Amount INTEGER :=32767
Position INTEGER := 1;
Buffer RAW(32767 ) ;
BEGIN
SELECT picture INTO Lob_loc FROM grape
WHERE grape_name = 'chardonnay' ;
DBMS_LOB.OPEN(LOB_loc, DBMS_LOBB.LOB_READONLY);
DBMS_LOB.READ(Lob_loc, Amount , Position , Buffer);
DBMS_LOB.CLOSE(LOb_loc);
END ;
here the file will be loaded in the buffer. The BFILE is manipulted through LOB locator.
You can also modify CLOB using
DBMS_LOB.WRITE(lob_loc,amount,offset, newtext ) ; here the newtext is added at position lob_loc + offset value.

delicious icon   digg icon   technorati icon   furl icon   |   2:08 PM | |

 

Multimedia Object Size

The size of multimedia data can make storage, processing, transmission and recption of data very costly.
    There are two approaches to cop with multimedia data size:-
  • Store data externally to database and you may just store metadata in database

  • Reducing size of multimedia data so that they can be stored within database

Object-relational databases offers means of stoing and processing multimedia data, based on SQL:1999. It provides two new data types that can hold media data.th first is "large object" oor LOB data type. THis has two variants, BLOB "binary large object " and CLOB Character large object. Oracle9i supports another datatype for media objects that are stored externally as operating system files (BFILE data type. LOBS in Oracle9i can hold up to 4GB size. A LOB can be broken down to chunks for processing. It also facilitates storing URLs containing audio, image, or video data stored on any HTTP server such as Oracle Application Server, Netscape Application Server, Microsoft Internet Information Server.

for more informaion here. Oracle sql synatx can be found here .

Reducing Media Object's Size

  • Signifiacnt reduction in media oobject size can be achieved by data compression using varios algorithms. But while doig compression we have to keep in mind that how much quality loss (in case of loosy compressions) is acceptable while decompression. In an application we can store data in one compression format and transform data in another while transmitting or presentation, so we need to store information about this. IBM's DB2 and Oracle allows us to do so by providing DB2IMAGE and ORDIMAGE types respectively. So we can store following attributes :
  • Compression format- such as jpeg,GIFLZW
  • Compression Quality value such as LOWCOMP ,HIGHCOmp
  • Content format value such as image type/pixel/data format
  • FileFormat - such as BMP, TIFF
high quality compression is obtained by using MPEG standards.

delicious icon   digg icon   technorati icon   furl icon   |   11:04 AM | |

 

Why need Multimedia Database

Data increasingly means not just numbers and small strings but multimedia data as well – structured text, images, video, audio, VR, etc.
As
  • multimedia data is large and will affect the storage , retrival and transmission of multimedia data.

  • multimedia data structures are completely different from standard database
  • In multimedia data structures we want content-based searching

  • In case of video and audio databases time to retrive information may be critical ex (Video on demand)


Multimedia Database Management System. (MMDBMS) must support multimedia data types in. addition to providing facilities for traditional DBMS like database creation, data modeling, data retrieval, data access and organization, and data independence.

Types of Multimedia datatypes :
  • Text (arbitary long text fields)

  • Still images

  • Graphics objects like CAD/CAM objects

  • Video data

  • Audio data

delicious icon   digg icon   technorati icon   furl icon   |   10:24 AM | |

 

Literature Survey on multimedia database

As i doing an literature survey on multimedia database for my seminar under Prof N. L. Sarda. So as go on with it i will post here what i learn.

delicious icon   digg icon   technorati icon   furl icon   |   10:07 AM | |