Click here to get back home

Serving Large Data From Files or From Blobs
 Home | NewsGroups | Search | About
Login Password
Register Now!
 comp.lang.perl.misc    Post an article   get this group's latest topics as an RSS feed add this group's latest topics to your My MSN content add this group's latest topics to your My Yahoo content
Subject Author Date
Serving Large Data From Files or From Blobs FeelLikeANut 02-15-2008
Get Chitika Premium
Posted by FeelLikeANut on February 15, 2008, 12:44 pm
Please log in for more thread options
 My question is partly a language issue (memory) and partly a Web issue
(where the program runs), so I hope everyone will be open to helping.

Users may submit and store pictures with my program. I have the choice
of storing these pictures either as files on the server or as a blob
in the database. The pictures are just user data, and it seems logical
to keep them in the database with all the other data. But I'm
concerned about how to serve these pictures. I can write a program to
handle image requests and return the appropriate picture from the
database. But it seems to me this means the picture will need to be
read from the database and stored in memory, in a variable, then
written from the variable to the HTTP response.

The part where the whole picture is stored in a variable is the part
that worries me. Is this impractical? Or merely a bad idea? Is storing
the pictures as files, allowing the server to serve them as it would
server anything else, the best option?

Posted by xhoster on February 15, 2008, 1:00 pm
Please log in for more thread options
 FeelLikeANut@gmail.com wrote:
> My question is partly a language issue (memory) and partly a Web issue
> (where the program runs), so I hope everyone will be open to helping.
>
> Users may submit and store pictures with my program. I have the choice
> of storing these pictures either as files on the server or as a blob
> in the database. The pictures are just user data, and it seems logical
> to keep them in the database with all the other data. But I'm
> concerned about how to serve these pictures. I can write a program to
> handle image requests and return the appropriate picture from the
> database. But it seems to me this means the picture will need to be
> read from the database and stored in memory, in a variable, then
> written from the variable to the HTTP response.

Databases that support blob types generally support ways to either stream
those blobs or efficiently read them in chunks. How to do that would
depend on the specifics of the database being used.

How are you getting the data into the database in the first place without
having it all in memory at once?

> The part where the whole picture is stored in a variable is the part
> that worries me. Is this impractical?

It depends on the maximum practical size of the image, and the amount of
memory your server has. Only you can know those things.


> Or merely a bad idea? Is storing
> the pictures as files, allowing the server to serve them as it would
> server anything else, the best option?

Do the file contents need the same transactional behavior (ACID, etc.)
as the rest of the stuff in database?

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

Posted by Rik Wasmus on February 15, 2008, 1:17 pm
Please log in for more thread options

> My question is partly a language issue (memory) and partly a Web issue=

> (where the program runs), so I hope everyone will be open to helping.
>
> Users may submit and store pictures with my program. I have the choice=

> of storing these pictures either as files on the server or as a blob
> in the database. The pictures are just user data, and it seems logical=

> to keep them in the database with all the other data. But I'm
> concerned about how to serve these pictures. I can write a program to
> handle image requests and return the appropriate picture from the
> database. But it seems to me this means the picture will need to be
> read from the database and stored in memory, in a variable, then
> written from the variable to the HTTP response.
>
> The part where the whole picture is stored in a variable is the part
> that worries me. Is this impractical? Or merely a bad idea? Is storing=

> the pictures as files, allowing the server to serve them as it would
> server anything else, the best option?


Insertion & retrieving is very much streamlined (huhu, pun not intended,=
=

but it indeed uses streams) with PDO, look at the Large Objects (LOB) =

example at <http://nl.php.net/pdo>:

Insertion:
<?php
$db =3D new PDO();//put your connections variables here
$stmt =3D $db->prepare("INSERT INTO tablename (image) values (?)");
$fp =3D fopen($_FILES['file']['tmp_name'], 'rb');
$stmt->bindParam(1, $fp, PDO::PARAM_LOB);
$stmt->execute();
fclose($fp);
?>

Retrieval:
<?php
$db =3D new PDO();//put your connections variables here
$stmt =3D $db->prepare("SELECT image FROM tablename WHERE id =3D ?");
$stmt->bindParam(1,$_GET['id'],PDO::PARAM_INT);
$stmt->execute();
$stmt->bindColumn(1, $lob, PDO::PARAM_LOB);
$stmt->fetch(PDO::FETCH_BOUND);
header("Content-Type: image/png");//or other format...
fpassthru($lob);
?>
-- =

Rik Wasmus

Similar ThreadsPosted
Large Data files and sizes - what are you doing about them? March 9, 2006, 1:43 pm
Printing Data usinf Perl to an HTML does not work with large amount of data December 6, 2004, 11:36 pm
perl and blobs March 1, 2005, 12:36 pm
comparing large number of data January 6, 2007, 10:52 pm
Help with downloading large files April 15, 2005, 1:31 pm
processing large files April 6, 2006, 2:42 pm
cgi upload of very large files (>2GB) April 25, 2006, 4:10 am
Opening large data file with Perl April 15, 2006, 1:14 am
lightweight access to large data structures? June 18, 2007, 5:20 pm
Help: joining 5 LARGE text files August 19, 2004, 4:05 am

Our other projects:

Art Dolls, Fairies and Mermaids - Sunnyfaces.net

Roy's Linux, Programming and Search Engines messages

1-Script XML SitemapXML Sitemap