The media server is now hosted on Google Code Project Hosting. You can access the full repository of changes and see the evolution of the project since 2007!
http://code.google.com/p/monolithicmedia/
Enjoy!
I have been hard at work the last few days on the Monolithic Media Server. There is an array of new features that makes programming more functionality in to the media server easier than ever.
Drupal and modular coding
Drupal has a lot of great benefits such as modular coding. What this means is ‘modules’ can be loaded in to the system by a click of a button. All the functionality of this module is dynamically loaded, other modules may depend on the functionality of other modules, but it can be removed at any time and the core system will still remain completely functional. Essentially, every extension can be view as a module, including themes and different ways of viewing data through a cell phone or Andriod App. All that supporting functionality is a feature of modular coding.
I have adapted this paradigm throughout the entire site. The more work I put in to it, the more stable the site as a whole becomes. I can make a single correction to the code and suddenly the entire site works as intended. This has always been my intention for the site.
Dependencies
A great feature of Drupal and my media server is its dependency support. Modules can now depend on each other and provide code for detecting libraries. When these libraries are missing, the module is disabled and the site can continue to function properly. I have created a status page for displaying information about such dependencies. Modules simply need to implement the status_ function and it can print out information to the status page.
Settings
Settings are another excellent new feature. In some cases, the module can act differently, based on the setup you are using, but you don’t necessarily want the end user to be able to decide how a module works. Therefore, modules can implement the configure_ function for displaying all of a module’s settings. Setting_ functions are used to validate setting input and set the defaults.
One nice feature about this is it makes it easy to install. Since all of the default settings are available, the administrator only has to view and set the settings they care about.
Easy module creation
This new API has made module creation extremely productive and easy. I have completed a large set of modules already, and there are plans in place for a dozen more. If you would like to contribute, please get in touch with me as I could really use the help. I’d like to finish by talked about some of the great new modules in the works!
Session management
The administrator will now be able to choose whether to use the default session manager, or a database session manager. The database session manager will support the load balancer, as well as provide quick and easy access for site administrators.
Load balancer
This module allows the administrator to set up additional mirrored servers for even faster and more efficient media access. One of my goals for this site is to make it completely scalable. The load balancer will redirect users to other mirrored setups for faster encoding, more efficient database access, and localized content. Redirects can take place based on any set of criteria. The balancer uses a waterfall effect, transferring the user to the last server (most matched criteria).
Web Dav
I am in the process of building a WebDav module that will allow access to all the media files loaded in to the database. It will use the database as a virtual file system, and will even allow an additional portal to user contributed content, practically replacing the upload module.
New tools
There is a vast array of tools out there. It is my goal to provide all the functionality of popular music organization programs such as Tag & Rename within the website interface. I plan to port over all the functionality of MythTV’s Jamu, TMDB, and TVDB scripts. All this functionality will be available to administrators. I plan on providing a Synchronization plug-in that will allow administrators to synchronize multiple media servers together, merging and selecting media files intelligently, and optionally, based on the musicbrainz.org database.
I am currently in the process of converting the entire media website over to a brand new API. This API should allow for even easier development and modification of the site.
New Features:
Performance has been improved with changes to the cron.php script that loads information about files. New options have been added for turning off recursive calls to display directories on sites whose file structure doesn’t change very often. The Live template has been improved greatly, adding more accessibility to all the functionality of the site. Development of a new JS template has begun with the inclusion of PHP-EXT, an open source project for generating EXT-JS code based on PHP objects. The settings API is nearing completion; this will allow users to save template information or global site options in their profile.
Templates:
The smarty template system has been removed because it's ugly, doesn’t abstract code very well, and has annoying structures for manipulating small amounts of information. The new template system uses PHP, which is prettier and more useful, while still offering a lot of abstraction.
Functional templates allow for blocks or pieces of the site to be included anywhere on the page. The new templates include features like quick-lists for file downloading, a brand new Administrative section for enabled and disabling templates and plug-ins, and new interfaces to the most popular plug-ins. Now, templates do not need to generate all the possible links that could be needed. Instead, plug-ins (such as encode, convert, and search) can display a default configuration page, or an “advanced search” page.
Administrative:
The newest version includes a brand new installation process for easy setup. There are thorough instructions for configuring each step, and it generates a settings.php file for you! It also checks all of the dependencies. A new Admin Tools section now supports loading tool plug-ins and a brand new menu system.
Security:
This new API comes with several new security features. Creating errors on the site is prevented by the new request processing that completely removes all variables that are not validated. Cross-site-scripting is prevented by a new register_output_vars() function that encodes all variables used in the output templates to be HTML compliant.
I have picked a version number based on its current progress. It is ready for download and development by the community! I have labeled it 0.4.0 (Alpha) because there is still a ton of work to be done. Fortunately, there is enough code already made and it is so fully featured it is just a matter of plugging out some code for more features. It is even cross platform! That’s right, it should now be able to run on WAMP (Windows, Apache, MySQL, PHP), LAMP (Linux, etc.), and MAMP (Mac, you get the point).
Current status and Feature overview:
It can run without a database, and just show a flat file structure (I don’t recommend running this unless it is behind a secure connection). Or it can run with a database and allow for quick searching! Aliasing can be used on paths by matching the paths with regular expressions, so even if your operating system doesn’t support links, the users can still see pretty paths! As discussed before, the site contains 3 main parts, “modules” which are responsible for handling different file types. It currently handles zip archives very nicely, thanks to the getid3() library for PHP. It also handles ISO images, and can catalog the entire file-system contained in an ISO. It supports any audio and video type that VLC can handle using an external process. It can convert any image that ImageMagick can handle (which is a ton!). Which leads me to the “plug-ins” section; plug-ins allows users to access the files. Plug-ins provides a link between the files loaded by the modules and the output of the files in the templates. The “encode” and “convert” plug-ins read files from the database and send them to other external programs, or set header information and allow for downloading. The “select” and “list” plug-ins are the most important; select allows users to view the files and select files to be downloaded in the future. The list plug-in allows users to download the list of selected files in many different formats. List formats can be defined in various ways, either a plug-in could be written to handle a specific type of list, or the template system can be used for outputting a type of list. Finally, it uses Smarty for its template system, so its appearance has limitless potential. There is default template that provides easy viewing on phones, and an EXT JS template that takes advantage of the EXT JS library to provide a beautiful Windows-like front-end to the users.
Plans:
Here I will discuss some of the plans for features I would like to add to the site. A “movies” module that can recognize DVD images and full length movies on the file system, I would like it to download information from IMDB. A “subversion” module that can recognize subversion databases on the file system, and alternatively checked out projects from a remote repository, it will then list all the files apart of the remote or local subversion, it will also be able to recognize diff files and versions. An archive plug-in will be added to handle outputting any file into multiple different archive formats. It will also handle outputting single files that come from archives. More templates! I plan on copying templates from more popular (but less feature-rich) services such as Ampache, Jinzora, kPlaylist, and even WinAmp Remote! Adding templates is easy using the Smarty template system.
Download:
Visit the download page for more information!
Links:
My progress on the media server website has been slow, but nevertheless it is coming along. The framework and backend part of the site is reaching the public testing stages and the source will soon be available for hacking.
As I mentioned in previous posts, the site revolves around 2 main sections of functionality. The ‘modules’ section which contains the input processors for reading in files, and the ‘plugins’ section which contains very specific functionality for using the files. The information is stored in a generalized database with each table containing the data for each individual module. The front end can consist of various templates.
Aesthetically pleasing design:
I have stopped working on the inferior but still useful list output, and have implemented an API called Ext JS that offers a wider range of functionality. Although it uses some intense JavaScript, the library is very stable and has turned out nicely. The best part is; I didn’t have to design any of it graphically. All I have done is implement the conceptual ideas and some extra functionality.
I am trying to accomplish a design that combines features from our favorite operating systems and media players. The windows like look and feel of a file browser, the iTunes look and feel of a media player, and a VLC conversion engine on the backend is the ultimate goal. As you can see from the site below, it is coming along nicely.
http://209.250.30.30/mediaserver/
Please save the security testing until the code is available, I am constantly making improvements on the backend security, but I can only work so fast.
Overview:
I have made a lot of progress on my media server website. I finally came up with a method of abstraction that I am happy with. I have the site organized into modules, which handle files of different types, and plug-ins, which handle displaying different aspects of each module.
Modules:
Modules handle files of different types, for example audio files or video files. Every module is extended from the “files” module which just stores general file information. Every module has some requirements that they must override, for example the handles function which takes a file as input and determines if the module can handle that file.
Plug-ins:
This contains the core functionality of the site. The plug-ins includes things like RSS feed, the file getter, and the viewer. The plug-ins can interact with each other, in which case they would have “dependencies”. Plug-ins are mainly responsible for display data that the modules collect.
Other Features:
The site will have strong template support. And if a file is not found for a template, then the default template file will be used. The appearance should mainly depend on the CSS template. There will also be a nice admin section.
This is a task that I have redone and return to many times. Here are some of the experiences I have gained when trying to accomplish this feat again.
Problem:
The ability to access my media (music, video, pictures) from any computer or device that I currently own. This is made more difficult because I keep expanding my array of devices, so I need something can I can use on any of them.
Goals:
I want something that can run on multiple devices with completely different environments driving them. I have a Treo with a very small touch screen, an iPod Touch. Both devices have a web-browser and supporting internet connection that will make it possible for me to access my dream site.
The iPod and Treo do not have mouse pointers which make some features like the MouseOver event unnecessary or impossible.
I want a site that will enable me to manage my media and view it in different ways. I want to combine features from file management systems, and add aesthetically pleasing interfaces to that.
Possible Solutions:
I have seen many file managers out there. Many of them have the features I am looking for as far as managing files goes. You can upload files, re-organize, create, and download. But all of them are lacking features that are critical for my server to be useful. For example, media type recognition. My application should recognize which files are downloadable and which files are viewable by the browser.
Microsoft Live and Folder Share. This is an excellent solution for viewing files on remote platforms. I cannot praise it enough. Not to mention it is so pretty! The icons are beautiful and the mouse over shows a nice effect. It is fast despite all the information transferring it has to do. But you can’t move around and manage the remote files through the interface.
Resolutions:
I will never have a website that does everything. But I should still be able to add onto it as time goes on and technologies or libraries change.
A concept that I really took into realization is from my Game Development class, Model/View/Controller. This can definitely be applied here. Separate your data from how it is displayed.
Abstracting away concepts of how other people will use my website, and how I will use my website. I want to be able to manage files, but no one else. Other people can view my files, but only some. I have resolved to not deal with this issue currently.
The site should select how to display something, as opposed to being hardcoded into a specific system. Each module should be responsible for handling its type of data. The modules should handle how to display their type of data.
Modularity/Abstraction:
I have always supported open source. And I plan to post my application on my website as well as a community development site (yet to be determined. suggestions?). A key requirement for making this site useful to others is abstracting my code so that it can be understood and modified by others.
The media server website should have to ability to be changed and added to. For example, I may only need to view music, pictures, and videos, but another person may want to view source code files or compressed files or Word documents. The site should be modular enough so that another programmer (future me) can add a module for viewing those types of file.
My solution for this is simple. I call each different module to handle a file, each module determines if it wants to handle the file otherwise it quits. Input is handled using only a file path. That should be a sufficient amount of information for it to read the file and determine some properties and store it however it needs to.
Output is the fun part, and I have a pretty good idea on how I want to handle that. Each module will be called and a priority system will be used. For example I have my Pictures module that can handle all picture file types, but my browser cannot view certain picture types like Photoshop PSD files. So I could have a Browser_Pictures module that will determine whether to display the picture or just download the picture when it is accessed. So Browser_Pictures replaces the standard Picture module when it comes to viewing.
Mobility. I need to be able to use this site on my devices, so I have come up with a system that will enable me to view the site in different ways. I will use a numeric value to determine the level of graphical displaying. Each module will of course be responsible for outputting the right way. The values will range from 1 which will display the files in a manner compatible with text based browsers, similar to the apache file list. The highest visual output is 10 which will hopefully be a nice mesh between iTunes, Windows Media Player, Windows Live, Vista, and OS X. Somewhere in there, there is a nice way to display files of different mediums.
Displaying (cont.):
Since this system is abstracted and modular it will be easy to display data in different way. I would like to have an RSS feed that will show recently added or modified files. Or a torrent file or everything selected compressed into a zip file or simply a link to a VLC server that is started at the same time. My solution for this is again allowing each module to handle its type of file. So when a file is accessed the module will output the appropriate links that it can understand for generating other content.
Again, since the site will be selective, a user can chose to display files in any way, even if it isn’t compatible with their browser. The modules will also support different modes of displaying, for example a Details view, as opposed to Tile view. The individual modules should understand the different type of view, and if they don’t then of course a default view will be used.
Also I love AJAX, so my website will be able to display little parts of media, instead of having a huge folder tree loaded all at once; it can add folder contents when it is clicked on. Very handy when you have the amount of media I do.
Feature List:
- Supports Music, Image, and Video files by default.
- Create playlists compatible with Windows Media Player and iTunes.
- Use the RSS feed to view new or modified files.
- Reads and displays compressed files and directory structures.
- Displays Subversion files and directory structures.
- Supports downloading using BitTorrent, Compressed Files, or through the browsers download manager.
- AJAX support for quick access on large servers.
- Supports all the VLC protocols for playing media, also includes playing media using browser plug-ins like Quick Time, Windows Media Player, and Flash.
- Can be used on Smart Phones, iPod Touch, iPhone, or any device with a browser.
- Can be tailored to fit any system that contains files you wish to access remotely.
- Has administrative features that allow more detailed management of files.
- Access your recorded TV shows, videos, pictures, and music from anywhere in the world.
Conclusion:
I have created a method for putting data into the database. Now I have to get data out. I have a lot of work ahead of me.
To Be Continued...