Do you have a question? Post it now! No Registration Necessary. Now with pictures!
- Posted on
- preg_match(_all) & big strings
September 2, 2004, 1:25 am
rate this thread
try to preg_match something in this I get always FALSE!
I have <File>....</File> tags in XML. Between tags is files contents
BASE64 encoded! When xml contains only one big over 4MB file in it,
preg_match and preg_match_all doesnt find it! When included file size is
nelow 4MB, preg functions works perfectly!
But PHP manual says that there is no size limitations to strings?!?!
Is this somekind BUG or why doesnt preg work over 4MB matches??? Looks
like problem is when patern-matched string is over 4MB, not the string
where to search!?!?!
How to get work this function? I have portal what should work with large
XML files (contains one ore more BASE64 coded files with unlimited
sizes). How can I get contents of those tags from XML???
/Sorry my bad english, my native language is estonian/
Re: preg_match(_all) & big strings
> I have problem with large textfiles! When I load over 4MB xml and
> try to preg_match something in this I get always FALSE!
> I have <File>....</File> tags in XML. Between tags is
> files contents
> BASE64 encoded! When xml contains only one big over 4MB file in it,
> preg_match and preg_match_all doesnt find it! When included file
> nelow 4MB, preg functions works perfectly!
> But PHP manual says that there is no size limitations to
> Is this somekind BUG or why doesnt preg work over 4MB matches???
> like problem is when patern-matched string is over 4MB, not the
> where to search!?!?!
> How to get work this function? I have portal what should work with
> XML files (contains one ore more BASE64 coded files with
> sizes). How can I get contents of those tags from XML???
> /Sorry my bad english, my native language is estonian/
If all else fails, why donít you try ereg functions. I think preg
and ereg are written by different developers, so they probably have
If that fails, the other option is to read the file and parse it into
an array, and do a regex by looping thru the array.
http://www.dbForumz.com / This article was posted by author's request
Articles individually checked for conformance to usenet standards
Visit Topic URL to contact author (reg. req'd). Report abuse:
Re: preg_match(_all) & big strings
First question is why are you trying to parse XML with regular expressions,
instead of an XML parser? You can't create a single regular expression capable
of parsing arbitrary XML as they aren't capable of maintaining enough state
information. They'll work for simple cases, but isn't that defeating part of
the purpose of XML?
http://www.php.net/manual/en/ref.xml.php uses Expat, which streams through the
XML file in manageable chunks, and may be more appropriate for your usage. I
think you'd just set up a handler and watch for <File> elements, and set up a
character data handler for the contents.
Well, given that preg_replace is part of the PCRE library, it might be worth
looking directly in the PCRE docs, which say:
The maximum length of a subject string is the largest posi-
tive number that an integer variable can hold. However, PCRE
uses recursion to handle subpatterns and indefinite repeti-
tion. This means that the available stack space may limit
the size of a subject string that can be processed by cer-
OK, so that appears to say for very simple expressions that you can probably
handle about 2GB of data, but that goes down depending on how complex your
expression is - possibly going down a lot, since it's based on "stack space",
if it's referring to the normal stack space that the OS provides processes.
What pattern are you using?
Hang on - I think I'm reading that incorrectly, but are you saying you're
supplying a 4MB _pattern_? A pattern that large is pretty absurd, and then
there's this from the docs:
The maximum length of a compiled pattern is 65539 (sic)
bytes if PCRE is compiled with the default internal linkage
size of 2. If you want to process regular expressions that
are truly enormous, you can compile PCRE with an internal
linkage size of 3 or 4 (see the README file in the source
distribution and the pcrebuild documentation for details).
If these cases the limit is substantially larger. However,
the speed of execution will be slower.
Another point to look out for is PHP's memory_limit directive - if you're
loading the whole lot into memory and processing it you're likely to hit that
limit. That's probably another point where the XML parser functions above will
win out, as you can read it in bits and write it off into a file rather than
having the whole lot in memory at one time.
<http://www.andyhsoftware.co.uk/space Space: disk usage analysis tool