Matte: Extending media support

Matte can be easily extended to support additional media types.

Matte can handle any type of file by way of one main Java API: magoffin.matt.ma2.MediaHandler. This API is responsible for reading files of a specific type, extracting metadata from those files, and handling requests for those files (possibly re-sizing them or re-compressing them along the way).

This API provides the mechanism for Matte to read arbitrary files and handle requests for those files. Two related APIs are the MediaRequest and MediaResponse.

The MediaRequest API encapsulates a request for a specific media item for specific parameters such as size and quality. The API looks like this:

When a request for a media item comes into Matte, a MediaRequest object will be instantiated for that request and populated with values as specified by that request, i.e. the mediaItemId, size, and quality. The request is then passed to the appropriate MediaHandler implementation for that media item.

The MediaResponse API allows the MediaHandler to returned the processed request result. The result might be to simply return the original media file, or it might be a re-sized, re-compressed copy of that media file. What ever the result is, it is up to the MediaHandler to determine how to service the request, and to then return that result via this API.

The MediaRequest and MediaResponse are similar to the Java ServletRequest and ServletResponse APIs, but geared specifically to Matte. To return a media item result, the MediaHandler must write the result to the java.io.OutputStream returned by MediaResponse.getOutputStream().

MediaHandler createNewMediaItem()

The MediaItem createNewMediaItem(File inputFile) method is called by Matte when a new file is uploaded into the application. The job of this method is to read the file and create a new instance of MediaItem from the information available in that file.

Note that this method is not responsible for persisting the new MediaItem instance, Matte will handle that automatically. It just needs to create the new object, populate fields on that object it knows how to populate, and return that new object.

Generally, implementations of this method need to accomplish the following:

Use the DomainObjectFactory to create a new MediaItem instance.
Populate the width (if available), height (if available), and mime properties of the new MediaItem instance.
Populate any metadata attributes it can read from the file.
Populate the MediaItem itemDate property, if available from the metadata in the file.

For example, here is a sample implementation adapted from the JpegMediaHandler implementation included with Matte that uses the Java ImageIO framework to read information from JPEG image files:

import javax.imageio.ImageReader;
import magoffin.matt.ma2.image.ImageMediaHelper;
import magoffin.matt.ma2.support.SmetaMediaMetadata;
import org.springframework.core.io.FileSystemResource;

public MediaItem createNewMediaItem(File inputFile) {
	// 1: create new MediaItem instance from DomainObjectFactory
	MediaItem item = getDomainObjectFactory().newMediaItemInstance();
	ImageReader reader = null;
	try {
		reader = getImageMediaHelper().getReaderForFile(inputFile);

// 2: set width, height, and MIME type information
		item.setWidth(new Integer(reader.getWidth(0)));
		item.setHeight(new Integer(reader.getHeight(0)));
		item.setMime("image/jpeg");

// 3: populate metadata
		MediaMetadata resultMeta = new SmetaMediaMetadata();
		resultMeta = resultMeta.setMediaResource(new FileSystemResource(inputFile));
		List<Metadata> results = new ArrayList<Metadata>(resultMeta.getMetadataMap().size());
		for ( Map.Entry<String,String> me : resultMeta.getMetadataMap().entrySet() ) {
			Metadata meta = domainObjectFactory.newMetadataInstance();
			meta.setKey(me.getKey());
			meta.setValue(me.getValue());
			item.getMetadata().add(meta);
		}
		
		// 4: set item date from metadata creation date, if avaialable
		if ( resultMeata.getCreationDate() != null ) {
			Calendar cal = Calendar.getInstance();
			cal.setTime(resultMeta.getCreationDate());
			item.setItemDate(cal);
		}
	} catch ( IOException e ) {
		throw new RuntimeException("Exception reading image", e);
	} finally {
		if ( reader != null ) {
			reader.dispose();
		}
	}
	return item;
}

This sample was for an image file type, but the overall process is the same for any file type.

MediaHandler getFileExtension()

This method is used to determine a file extension for the media item based on a specific MediaRequest. For many MediaHandler implementations, the file extension for the media files it handles is static and does not change based on the request. For example a JPEG image handler might always return JPEG images, even after re-sizing or re-compressing the original file, so would always return "jpg" as the file extension.

On the other hand, some MediaHandler implementations might return different file types based on the parameters of the request. For example an MP3 audio file handler would want to return "mp3" as the file extension. But for all other requests, it might return an embedded album cover PNG image, and so would return "png" as the file extension for those requests.

MediaHandler getEffect()

Presently this method exists to allow for arbitrary effects to be plugged into Matte and made available for users to choose to be applied to their media items. For example a watermark effect would allow a user to specify a watermark be applied to all of their media items. However, at this time Matte does not support this. This method is still utilized by AbstractMediaHandler (discussed later in this document).

MediaHandler handleMediaRequest()

This is the method that does the work of responding to a MediaRequest for a media item, processing the media item according to the paremters of that request, and return the result on a MediaResponse. When Matte processes requests for items, it will use the MediaHandler implementation configured to process media items of the type specified in the request. So if a request comes in for a MediaItem that is a JPEG image, Matte will look up the MediaHandler implementation configured for JPEG images and call this method on it. Thus when this method is called, it can be assumed the request is for a file type that the handler knows how to process.

The general steps this method must perform are then:

Load the media item file requested by the MediaRequest. The MediaHandler can get a Spring Resource object for the requested MediaItem by calling the getMediaItemResource() method on the MediaBiz API.
Process the media item file to service the request parameters as much as possible, for example re-size the item to the size specified by MediaRequest.getMediaSize(), re-compress the item to the level of quality specified by MediaRequest.getMediaQuality(), and apply any MediaEffect objects specified by MediaRequest.getEffects().
Set the values on the MediaResponse that are appropriate for the result of processing the media item, such as the number of bytes in the result (MediaResponse.setMediaLength()), the MediaItem instance being processed, the MIME type of the response (MediaResponse.setMimeType().
Write the bytes of the processed media item result to the java.io.OutputStream provided by the MediaResponse.getOutputStream() method.

For example, here is a sample implementation adapted from the JpegMediaHandler implementation included with Matte that uses the Java ImageIO framework to process JPEG image files:

Matte built-in MediaHandler implementations

Matte comes with many implementations to handle a variety of file types already, and they form a class hierarchy that makes it easy to add new implementations without needing to write a lot of code.

Matte MediaHandler implementation class hierarchy

AbstractMediaHandler

The AbstractMediaHandler class is a good class to extend when implementing MediaHandler. It provides many methods useful for all implementations. It provides support for the concept of media effects, which are transformations applied to media items when they are requested. These can be anything such as re-sizing, rotating, or applying a watermark. The MediaEffect API defines a standardized way of applying effects, and AbstractMediaHandler defines some helpful fields and methods for configuring media effects and applying them.

The MediaEffect API if also pretty simple:

BaseImageMediaHandler, BaseAwtImageMediaHandler, BaseJMagickMediaHandler

The BaseImageMediaHandler class extends AbstractMediaHandler to provide some additional support for image-related metadata processing.

The BaseAwtImageMediaHandler class extends BaseImageMediaHandler to provide some additional support for Java AWT based media effects. The AwtMediaEffect API extends MediaEffect and makes it easy to apply effects to images using the java.awt.image.BufferedImage class for image manipulations.

Often the built-in Java AWT-based classes are slow when handling large image files and applying transformations. The ImageMagick library is a cross-platform image manipulation framework that is very good at processing images quickly and with very good quality. There is a Java wrapper around this library called JMagick. The BaseJMagickMediaHandler class extends BaseImageMediaHandler to provide some additional support for JMagick image processing and media effects. The JMagickMediaEffect API extends MediaEffect and makes it easy to apply effects to images using the magick.MagickImage class for image manipulations.

BasicIconBasedMediaHandler

Matte is designed to support any file type, not just image types. In order to display non-image file types to users, however, Matte requires an image of some type to be returned for every file type. The BasicIconBasedMediaHandler class helps here by supporting returning icons for files that aren't images. It can be configured to return icons based on the MIME type of a media item.

It also has the ability to return images that are embedded as metadata within the original media file. For example it is common to find album covers stored as PNG or JPEG images within MP3 files. The BasicIconBasedMediaHandler can be used to extract those embedded images and then pass off the handling of those images to a real image-focused MediaHandler delegate implementation, based on the embedded image's MIME type.

The EmbeddedImageMetadata API is used to signal that a given file has an embedded image available in it's metadata. Thus when handling a request, the BasicIconBasedMediaHandler can check to see if the MediaMetadata instance instantiated for that file implements this interface. If it does, then it will delegate the request to its configured MediaHandlerDelegate implementation. The MediaHandlerDelegate API is similar to the MediaHandler API, but simplified:

MimeTypeMediaHandlerDelegate

Since handling delegate requests is pretty much the same as handling a non-delegate request, it makes sense for some image MediaHandler implementations to also implement MediaHandlerDelegate. In fact the JpgeMediaHandler and PngMediaHandler classes to just that. Thus is would be easy to re-use these classes as delegate handlers, and the MimeTypeMediaHandlerDelegate class provides an easy way to do this. It can be configured with a mapping of MIME types to MediaHandlerDelegate instances, which can be the same JPEG and PNG MediaHandler instances configured in Matte.

This is what the default configuration for JPEG, PNG, and MP3 files looks like in Matte:

Notice how the mp3MediaHandler's imageMediaRequestDelegate is a MimeTypeMediaHandlerDelegate instance, and its delegateMap is configured with mappings for JPEG and PNG MIME types, delegating to the jpegMediaHandler and pngMediaHandler classes that are also the defulat JPEG and PNG MediaHandler instances used by Matte.