Page 2 of 7

Re: A bit of regex help needed

Posted: 23 Apr 2012 12:56
by highend
If the separator is a doublequote (because it marks the end of the url) instead of the & char, just modify your regex:
$type = regexreplace("$token", "(.+?(?=&|")).*", "$1");

Re: A bit of regex help needed

Posted: 23 Apr 2012 13:13
by tiago
No, it's not working.
An idea: first match all youtube links "http://www.youtube.com/watch?v=ZuGgm8UQ&something", then clean them down to "http://www.youtube.com/watch?v=ZuGgm8UQ" format using your regex. But how?

Is my version of your code right, by the way? :oops:

Re: A bit of regex help needed

Posted: 23 Apr 2012 13:17
by highend
Attach a sample file with the full structure including a few (good and malformed) links. Everything else is pure speculation (from my side). Post a sample how the result should look like.

Should an output look like this?
http://www.youtube.com/watch?v=abcWERF-W
http://www.youtube.com/watch?v=IFweFF-Q
http://www.youtube.com/watch?v=ZZqqcd-a
etc.

Re: A bit of regex help needed

Posted: 23 Apr 2012 13:26
by Stefan
Maybe something like this is needed?

Get position of the string "HTTP" and then of the very next space, that's the URL:

Code: Select all

  $A = "text1 text2 http://www.youtube.non/watch?v=vvvvv text3 text4 http://www.youtube.non/watch?v=Zu";
  $A = $A ."Ggm8UQ-7E&feature=g-all-u&context=G202c46eFAAAAAAAAEAA text5 text6";
  $Content = $A; //readfile();
  
  //concatenated lines, glue splitted URLs:
  $String = replace($Content, "<crlf>", " ");


  set $OUT;

  //as long as there is any string "HTTP".... do:
  while (strpos($String, "HTTP") > 0 )
  {
     //find pos of "http":
        $HTTPpos = strpos($String, "HTTP"); 
    //find pos of next space:
       $URLEND  = strpos($String, " ", $HTTPpos); 
    // get the whole URL:
       $URL     = substr($String, $HTTPpos, $URLEND - $HTTPpos); 
    //drop all behind the first found "&":  
        if (strpos($URL, "&") > 0)
        {
            $URL = substr($URL, 0, strpos($URL, "&"));
        }
    //create out array:
       $OUT = $OUT . $URL . "<crlf>";
    //remove URL from $String to get to an end:
       $String = substr($String, $URLEND +1);
  }

  //Test output:
         msg "Rest string: $String";
              //Rest string: text5 text6
         text $OUT;
              //http://www.youtube.non/watch?v=vvvvv
              //http://www.youtube.non/watch?v=ZuGgm8UQ-7E&feature=g-all-u&context=G202c46eFAAAAAAAAEAA
                  // WITH "drop all behind the first found "&""
              //http://www.youtube.non/watch?v=ZuGgm8UQ-7E



EDIT:
//drop all behind the first found "&":

Re: A bit of regex help needed

Posted: 23 Apr 2012 13:40
by tiago
A good sample with a confusing structure (4th link, 09:16).
Stefan: your code works on links provided by you, but hangs on the content below.
The following are some links from You Tube that you might find useful when teaching Science and Religion. Always watch the clip before using it in the classroom. If you are having problems using YT in your school then at the bottom of the page are some instructions on how to download, store and play YT clips off line.

Do you have any clips which you really like? Let me know

Some video clips
God Delusion: Richard Dawkins

John Polkinghorne Discusses Science and Religion

Alistar McGrath
Atheist turned Christian


Some extracts from the God Delusion read by RD.

John Polkinghorne, Physicist and Priest talks about the similarities of Science and Religion

An interview with AG author of the God delusion
07:16 - http://www.youtube.com/watch?v=rte3kfzNXPg

05:13 - http://www.youtube.com/watch?feature=pl ... 1tfWtCZB_A

08:55 - http://www.youtube.com/watch?v=DBRKY8Qx ... re=related

Mark Vernon : Science, Religion & the Meaning of Life

Science, Religion and the Cosmos

Daniel Dennett


An interview with the author about certainty and uncertainty

A personal view on science, the world and the place of religion

An interview with atheist biologist Daniel Dennentt
09:16 - http://www.youtube.com/watch?feature=pl ... sJEub2KsLo#!

03:56 - http://http://www.youtube.com/watch?v=J ... ure=fvwrel

09:45 - http://www.youtube.com/watch?v=LUqFUl8nS_g

Re: A bit of regex help needed

Posted: 23 Apr 2012 13:46
by tiago
yes, highend the desired output should go as
http://www.youtube.com/watch?v=rte3kfzNXPg
http://www.youtube.com/watch?v=e1tfWtCZB_A
and so on.

Re: A bit of regex help needed

Posted: 23 Apr 2012 14:21
by highend
Try this one:

Code: Select all

	$files = folderreport("files", "r", , , , "|");

	$re = "";
	foreach($file, "$files", "|"){
		$content = readfile("$file", "t");
		foreach($line, "$content", "<crlf>"){
			$type = regexreplace("$line", ".+?(http.+$)", "$1");
			if($type != $line){
				if(strpos($type, "?v") != -1){
					$type = regexreplace($type, "(.+?(?=&)).*", "$1");
				} else {
					$type = regexreplace($type, "(.+?\?)(.+\&)(.*)", "$1$3");
				}
				$re = "$re" . "$type<crlf>";
			}
		}
	}

	text $re;


Re: A bit of regex help needed

Posted: 24 Apr 2012 00:29
by tiago
highend, thank you!
Stefan, thanks too.

Re: A bit of regex help needed

Posted: 09 May 2012 16:31
by tiago
highend wrote:Try this one:

Code: Select all

	$files = folderreport("files", "r", , , , "|");

	$re = "";
	foreach($file, "$files", "|"){
		$content = readfile("$file", "t");
		foreach($line, "$content", "<crlf>"){
			$type = regexreplace("$line", ".+?(http.+$)", "$1");
			if($type != $line){
				if(strpos($type, "?v") != -1){
					$type = regexreplace($type, "(.+?(?=&)).*", "$1");
				} else {
					$type = regexreplace($type, "(.+?\?)(.+\&)(.*)", "$1$3");
				}
				$re = "$re" . "$type<crlf>";
			}
		}
	}

	text $re;

How could this detect youtube links if they are in the very first column of a line, too? It's the only case the script is failing and I can't see a way to fix it. Sorry for this.

Re: A bit of regex help needed

Posted: 09 May 2012 16:53
by Stefan
tiago wrote:How could this detect youtube links if they are in the very first column of a line, too?
The regex modifier + tries to match ONE-or-more of the previous expression.
Exchange them with * to to match NONE-or-more



.

Re: A bit of regex help needed

Posted: 09 May 2012 17:14
by highend
The regex modifier + tries to match ONE-or-more of the previous expression.
Exchange them with * to to match NONE-or-more
Replacing ".+?" with ".*?" in the $type = ... line alone doesn't do the trick.

In that case $line will be the same as $type and the next if command isn't executed.

A simple workaround would be:

Code: Select all

   $files = folderreport("files", "r", , , , "|");

   $re = "";
   foreach($file, "$files", "|"){
      $content = readfile("$file", "t");
      foreach($line, "$content", "<crlf>"){
         $type = regexreplace("$line", ".*?(http.+$)", "$1");
         if(strpos($line, "youtube") != -1){
            if(strpos($type, "?v") != -1){
               $type = regexreplace($type, "(.+?(?=&)).*", "$1");
            } else {
               $type = regexreplace($type, "(.+?\?)(.+\&)(.*)", "$1$3");
            }
            $re = "$re" . "$type<crlf>";
         }
      }
   }

   text $re;
This would work for all lines which contain the word "youtube". Feel free to change that to "http://www.youtube.com" or whatever you need.

Re: A bit of regex help needed

Posted: 09 May 2012 17:34
by tiago
Thanks guys!

Re: A bit of regex help needed

Posted: 28 May 2012 15:37
by tiago
Can one of you please check if these are correct?

Code: Select all

"Base Name WITH dot"
 $a = "v8.1.315 by PSpad.txt"; $bn = regexreplace($a, "(.+\.).*", "$1"); echo $bn;
"Base Ext - NO dot"
 $a = "v8.1.315 by PSpad.txt"; $bn = regexreplace($a, "(.+\.)(.*)", "$2"); echo $bn;

"Base Ext WITH dot"
 $a = "v8.1.315 by PSpad.txt"; $bn = regexreplace($a, "(.+)(\..*)", "$2"); echo $bn;
"Base Name - NO dot"
 $a = "v8.1.315 by PSpad.txt"; $bn = regexreplace($a, "(.+)(\..*)", "$1"); echo $bn;
I wrote other regexes in the past and in a later point I found some broken.

Re: A bit of regex help needed

Posted: 28 May 2012 15:55
by highend
As long as you're checking only windows files (unix files don't have an extension) and no directories, you should be fine.

Since gettoken() can be used to access tokens from the right side of a string it's an alternative to store the file extension as well (and it's faster for those operations).

Re: A bit of regex help needed

Posted: 28 May 2012 16:30
by tiago
Thank you!