threaded | unthreaded | newest first

phpPerson problem with too many friends??

topic posted Mon, July 11, 2005 - 1:39 AM by  robin

can anyone tell me why this profile seems to crash the php code ?

www.tribe.net/FOAF/342868...30545efc608d
posted by:
robin
offline robin
334 friends
join to post
  • Julian
    Julian
    offline 13

    Re: phpPerson problem with too many friends??

    Mon, July 11, 2005 - 6:12 AM
    Which php code (I guess it's probably mine)? And how does it crash? Any error messages?
    join to post
    • robin
      robin
      offline 334

      Re: phpPerson problem with too many friends??

      Mon, July 11, 2005 - 8:30 AM

      no error message.

      when i try it with you test it just returns nothing it seems.

      when i try it with mine i get a 500 error.

      unfortunately my ISP doesn't give me access to the error logs.

      still trying to troubleshoot it but it seemed when i tested with just your code there were some problems with it too..

      can you try to import that FOAF with your code on your server? i'd be interested to know if you get the data in a structure.

      thanks
      join to post
    • robin
      robin
      offline 334

      Re: phpPerson problem with too many friends??

      Mon, July 11, 2005 - 9:00 AM

      i found the code on your server and tried it with that foaf..

      www.voidstar.com/foafPerso...onTest.php

      seems to work..

      however the same code on my server..

      www.devweb.org/foafperson...sonTest.php

      i guess mine is timing out or something maybe?

      that's one guess.. hard to say with no errors..

      any other ideas?
      join to post
      • Julian
        Julian
        offline 13

        Re: phpPerson problem with too many friends??

        Mon, July 11, 2005 - 11:23 AM
        Works for me here.
        tinyurl.com/dt8ag

        If you don't see meaningful error messages, try adding some print "hello" statements and perhaps printing out the output at various stages. eg Is foafPerson_FetchDataFile actually returning any data? Perhaps fopen is disabled.
        join to post
        • robin
          robin
          offline 334

          Re: phpPerson problem with too many friends??

          Mon, July 11, 2005 - 11:54 AM

          it works with any other FOAF file I have tried so far.

          like mine.

          www.devweb.org/foafperson...sonTest.php

          it just doesn't work with this one:

          www.devweb.org/foafperson...sonTest.php

          the only real difference that is obvious is the volume of friends.
          but maybe there is something else i am missing.




          join to post
          • matt...
            mattpaul
            offline 43

            Re: phpPerson problem with too many friends??

            Tue, July 12, 2005 - 12:48 PM
            how many friends are there in that troublesome FOAF?

            looks like your test rig first runs it through printf / debug like code (array[1] etc.); and then through a second analysis at the bottom....

            I have yet to peek at any of the FOAF reader code but if there is a problem due to size of the dataset that suggests to me that the reader code is trying to construct an in memory representation of the entire FOAF file at once.

            this is an intuitive approach to processing a graph and often tempting to implement but in some runtime environments it might be inappropriate.

            the other type of approach would be to implement a buffered stream reader in which you ask for a lil' more of the data at a time instead of receiving one fat data struct populated with all of the FOAF's data. you could have a function to fetchFOAFProfile() which would return user profile data, and then iterator like functions to work with the potentially long list of "knows" relations.... firstFOAFKnows(), nextFOAFKnows(), nextNFOAFKnows(), lastFOAFKnows() - horrible function names I know(s)!!! ugh. feel free to suggest improvements!

            mattpaul
            join to post
            • robin
              robin
              offline 334

              Re: phpPerson problem with too many friends??

              Tue, July 12, 2005 - 1:10 PM

              it does seem to be something like that..

              i haven't really started to go through all the provided code to see where the exact problem happens.

              i'm actually upgrading my server soon and hoping that might help as i don't see any reason why the same code would work on Julian's server but not mine.

              here's my phpinfo.. perhaps it holds some answers?

              www.devweb.org/phpinfo.php

              can you post one for your server, Julian?
              join to post
            • Julian
              Julian
              offline 13

              Re: phpPerson problem with too many friends??

              Wed, July 13, 2005 - 12:50 AM
              My foafperson code is built on top of RAP which is a PHP RDF parser. The nature of RDF means that you have to build a complete model of an RDF file before you can start interpreting it. RAP does have a DBMS based model store but for our purposes it's rare for a FOAF file to grow so large that it's necessary. I've run a FOAF file with 10,000 friends through it and although it was horribly slow it did complete.

              I suppose it's possible that my output FOAF array is getting too big and hitting some php resource limitation but I doubt it. i'm not going to post a phpinfo trace here as there's rather too much info that shouldn't be given away. but key factors maybe
              memory_limit 16M
              max_execution_time 900
              join to post
              • Julian
                Julian
                offline 13

                Re: phpPerson problem with too many friends??

                Wed, July 13, 2005 - 12:55 AM
                Also, If I was debugging this, I'd go through and comment out blocks of function. Try and separately answer
                - Is the file being collected ok?
                - Is RAP parsing it and creating a triple store?
                - Is the FOAF interpretation code completing?
                - Is it just tming out?

                After that we may be into updating RAP and modifying the code to support whatever the latest version is.

                I have to say that this was last year's project and I'm not running any production system that depends on it, so I'm not overly motivated to sort it out. Like all my public code, it's public domain, do whatever you like with it.
                join to post
                • matt...
                  mattpaul
                  offline 43

                  Re: phpPerson problem with too many friends??

                  Wed, July 13, 2005 - 11:05 AM
                  these are good places to start debugging if you haven't yet, Robin.

                  btw: what is a triple store? sounds like a street fighter move :)

                  configuring RAP to use a RDBMS does sound a tad overkill for FOAF! I think most likely Robin is hitting the web environment's limits. I'm not that familiar with how fast PHP code bloats but I know Perl's dynamic arrays and hashes can quickly grow to ungodly size.

                  thnx & best wishes carrying the FOAF torch,
                  mattpaul

                  p.s. certainly didn't mean to harsh on your public code - thanks for sharing!
                  join to post