PHP Basic Tutorials ,PHP Sources Code,PHP Online Example

Algorithms Arrays Passwords Uncategorized
Commerce Uncategorized
Databases Abstraction Layers Informix LDAP MS SQL MySQL ODBC PostgreSQL Uncategorized
Date & Time Calculators Conversion Translation Uncategorized
Discussion Chat Forums Guestbooks Polls Uncategorized
Email Validation Uncategorized
External Uncategorized
Files and Directories Uncategorized
Games Uncategorized
Graphics 3-D Graphs Resizing Thumbnails Uncategorized
HTML Forms Menus & Navigation Source Viewers Tables Templates Uncategorized
HTTP Authentication Cookies File Uploading Uncategorized
I18N Uncategorized
Math Uncategorized
Miscellaneous Uncategorized
Searching and Trees Uncategorized
Security Uncategorized
Text Uncategorized
Utilities Calendars Counters Link Checkers Weather Uncategorized
XML Uncategorized

php sample source codes
ananse is web crawler that collects links from a website, crawls them and indexes the data into a database for use in searching. This version "ananse v1.1" is the second experimental version of the real "ananse" project. Developed by Devonet Multimedia, Accra, Ghana. http://www.devonet.com /* Ananse v1.1 is still under development. This is just the second experimental module of the "ananse" project. Developed by Cornelius Duhadzi, CEO and Creative Director of Devonet Multimedia, Accra, Ghana. http://www.devonet.com Download ananse v1.1 at: http://www.devonet.com/free. Send your comments, questions and suggestions to mynelis@gmail.com / SET UP THE DATABASE ====================================================== Create a database named "mycrawler" Create a table called "ananse" in the "mycrawler" as below: CREATE TABLE `ananse` ( `acID` bigint(20) NOT NULL auto_increment, `acURL` varchar(250) default NULL, `acMetaDes` text NOT NULL, `acMetaKey` text NOT NULL, `acLink` text NOT NULL, `acTitle` varchar(250) NOT NULL default '', `acContent` longtext NOT NULL, PRIMARY KEY (`acID`) ) ENGINE=MyISAM COMMENT='ananse mini crawler'; SET UP THE URLs YOU WANT TO CRAWL (urls.txt) ====================================================== Put the URLs you want to crawl in a text file named "urls.txt". Must be in the same directory. Put each URL on a new line. SET UP THE THE CONFIG FILE(config.php) ====================================================== <? #general settings error_reporting(0); ini_set("max_execution_time", 0); #A: mysql database server login settings $dbhost = "localhost"; $dbdb = "ananse"; $dbuser = ""; $dbpass = ""; #B: mysql database storage details define('table','ananse'); define('id_field','acID'); define('url_field','acURL'); define('meta_description_field','acMetaDes'); define('meta_keywords_field','acMetaKey'); define('link_field','acLink'); define('title_field','acTitle'); define('content_field','acContent'); #connect to database server $conn = mysql_connect($dbhost, $dbuser, $dbpass) or die("Could not connect to server"); mysql_select_db($dbdb, $conn) or die("Could not connect to database"); ?> SET UP THE THE MAIN CRAWLER PAGE (index.php) ====================================================== <? //includes require_once("config.php"); //open the site for crawling function open_url($url){ if(fopen($url,'r')){ return file_get_contents($url); } else { return false; } }#end open url //collect links function collect_links($url){ $link = null; $site = open_url($url); $combined["url"] = $url; $combined['links'] = array(); preg_match_all("/<a.?href[=]['\"](.+?)['\"].*?>(.+?)<\/a>/i",$site, $link, PREG_SET_ORDER); for($s=0; $s<count($link); $s++){ array_shift($link[$s]); array_push($combined['links'], $link[$s][0]); } $combined['links'] = array_unique($combined['links']); $combined = array_unique($combined); return $combined; }#end collect links #get absolute path of link function locate($link, $url){ $path = $link; if(!preg_match("(http://\|www.)i", $link))$path = $url."/".$link; if(!fopen($path, "r")){ return ""; } else return $path; }#end get path //get meta tags function get_meta($link, $url){ $link = locate($link, $url); $meta = array('description', 'keywords'); $metatags = get_meta_tags($link); $meta['description'] = isset($metatags['description'])?$metatags['description']:""; $meta['keywords'] = isset($metatags['keywords'])?$metatags['keywords']:""; return $meta; }#end meta tags //get page title function get_title($link, $url){ $title = ""; $match = array(); $link = locate($link, $url); $content = file_get_contents($link); if(preg_match("/<title>(.+?)<\/title>/i", $content, $match)){ $title = isset($match[1])?$match[1]:$title; } return $title; }#end page title //get page content function get_content($link, $url){ $link = locate($link, $url); if(!empty($link)){ return file_get_contents($link); } else return ""; }#end page content //index all urls function index($url){ if(!fopen($url, "r")){ print "<br><h3><i>".$url."</i> could not be reached.</h3>"; flush(); } else { print "<script>window.status='Indexing: ".$url."';</script>"; print "<br><h3>Indexing <i>".$url."</i>...</h3>"; print "<ol>"; $collection = collect_links($url); $total = count($collection['links']); for($i=0; $i<$total; $i++){ $current = isset($collection['links'][$i])?$collection['links'][$i]:""; if(eregi(" ",$current))$current = ""; if(!empty($current)){ if(!get_content($current, $url)){ $reachable = false; } else { $reachable = true; } $meta = get_meta($current, $url); print "<li>".$current."</li>"; print "<script>window.status='Current URL: $url \| Current Position: ".($i+1)." of $total \| Current Link: ".$current."';</script>"; print "<script>document.title='Indexing ".$current."';</script>"; flush(); $matades = base64_encode($meta['description']); $metakey = base64_encode($meta['keywords']); $title = base64_encode(get_title($current, $url)); $content = base64_encode(strip_tags(get_content($current, $url))); #end encoding $sql = "select ".id_field." from ".table." where ".url_field."='$url' and ".link_field."='$current'"; $test = mysql_query($sql); $len = mysql_num_rows($test); if($len == 0){ $doSql = "insert into ".table; $doSql .= " (`".url_field."`,`".meta_description_field."`,`".meta_keywords_field."`,`".link_field."`,`".title_field."`,`".content_field."` )"; $doSql .= " values ('$url', '$matades', '$metakey', '$current', '$title', '$content')"; } else { $doSql = "update ".table; $doSql .= " set ".meta_description_field."='$matades'"; $doSql .= " ,".meta_keywords_field."='$metakey'"; $doSql .= " , ".title_field."='$title'"; $doSql .= " , ".content_field."='$content'"; $doSql .= " where ".url_field."='$url' and ".link_field."='$current'"; } flush(); if(mysql_query($doSql) and $reachable == true){ print " <b> OK </b>"; } elseif($reachable == false) { print " <b> FAILED </b>"; } flush(); } } print "</ol>"; print "<h4>Done</h4>"; print "<script>window.status='$url indexed successfully!';</script>"; print "<script>window.title='$url indexed successfully!';</script>"; } }#end index #start crawling if(isset($_GET['crawl'])){ $urlList = file("urls.txt"); if(isset($_GET['url'])){ $i = $_GET['offset']; $url = trim($urlList[$i]); } else { $i = 0; $url = trim($urlList[0]); } index($url); mysql_close($conn); if($i < count($urlList)-1){ print "<script>location='index.php?crawl&url=".$url."&offset=".($i+1)."';</script>"; } else { print "<script>location='index.php';</script>"; } } else { print "<a href=\"index.php?crawl\">Crawl Entire List</a>"; } ?>
Sort of a userfriendly var_dump, this function will list any requested variables it can lay its hands on, and collapse the list for you to expand at will. All global variables at the tip of a mouse. If I include this on pages that do not work correctly, the problem never stays hidden for long! <? $dump_count=0;$dump_flag=0; function dumpdata($v="GLOBALS") { if(!strstr($_COOKIE['pw'],'secret6')) return false; // for my eyes only global $dump_count,$dump_flag;$skip=array("HTTP_GET_VARS","HTTP_POST_VARS","HTTP_COOKIE_VARS","HTTP_ENV_VARS","HTTP_POST_FILES","HTTP_SERVER_VARS"); // skip those // strangely the $_SESSION vars are never made public, but the $HTTP_SESSION_VARS are available. echo "<blockquote id=\"dump$dump_count\" style=\"margin:0px 18px;".($dump_flag?"display:none":"")."\">\n"; $q="while(list(\$key,\$val) = each($".$v.") ) { ". " \$dump_count++; ". " if( \$key===0 \|\| !in_array(\$key,\$skip) ) ". " echo \"<b>$\$key</b> = \".@htmlspecialchars( (string) \$val).\" \".(is_array(\$val)?count(\$val)==0?\"0\":\"<a href=javascript:void((a=document.getElementById("dump\$dump_count").style).display=a.display=="none"?"block":"none")>\".count(\$val).\"</a>\":\"\").\"<br>\n\"; ". " if( is_array(\$val) && \$key != \"GLOBALS\"&& !in_array(\$key,\$skip) ) { ". " \$dump_flag=true;dumpdata( \$v.\"[\".\$key.\"]\" ); ". "}}"; eval($q); echo "</blockquote>\n"; } ?>
Search a database with this function. Tables and fields are automatically found and searched. Exclude words or table names if you like (no_search_words.txt, no_search_tables.txt). Please send comments to jim@jarzion.com or <a href='http://www.jarzion.com'>Click Here</a> <?PHP /************************************************************************* JHsearch() was written by James R Hartford JHsearch() may be used and modified freely, but I ask that you please note the original author (James R Hartford) and refer to the function by its original name (JHsearch()) If you do make changes that I may find useful, please email them to jim@jarzion.com and I will include your name in later versions of JHsearch() database connection is set up for MySQL but can be changed if needed. ************************************************************************/ //use these globals to set up your database connection. global $dbhost; $dbhost = ''; global $dbname; $dbname = ''; global $dbuser; $dbuser = ''; global $dbpass; $dbpass = ''; /*************************************************************** Below is an example of how the JHsearch function can be used and display the results of the search. You can test for the table name and display the result accordingly for that table. ******************************************************************/ $searchResult = JHsearch("test a"); if (count($searchResult) < 1){ $output .= "No results for search 'test a'"; }else{ $SR = array_keys($searchResult); for($i = 0; $i < count($SR); $i++){ if($SR[$i] == "site_info"){//this is checking to see if the table name is media for($c=0; $c < count($searchResult[$SR[$i]]); $c++){ echo "Siteinfo: ".$searchResult[$SR[$i]][$c]['description']."<br>"; } } if($SR[$i] == "class"){//this is checking to see if the table name is media for($c=0; $c < count($searchResult[$SR[$i]]); $c++){ echo "Class: ".$searchResult[$SR[$i]][$c]['title']."<br>"; } } } } /############################################################## ################################################################/ /******************************************************** Function: JHsearch($search_string) Purpose: search an entire database and return the results in a 3 dimensional array. Words found in no_search_words.txt will not be searched. Table names found in no_search_tables.txt will not be searched. result example: $result[table_name][row][field_name] Author: James R Hartford *********************************************************/ function JHsearch( $string ){ $newstring = $string; $searchcrit = explode(" ", $newstring); //get the words to be searched $dontsearchwords = JHgetnos("words"); //get the words not to search into an array //get the actual search criteria $searchcrit = array_diff($searchcrit, $dontsearchwords); $searchcrit = JHarray_reindex($searchcrit); //get the tables to search $query = "Show Tables"; $tables = JHRquery($query); for($i = 0; $i < count($tables); $i++){ $gettables[$i] = $tables[$i][0]; } $tables = $gettables; $tabletakeout = JHgetnos("tables"); //tables to leave out of search $tables = array_diff($tables, $tabletakeout); $tables = JHarray_reindex($tables); $resultcount = 0; //Build the search query for($i = 0; $i < count($tables) && count($searchcrit) > 0; $i++){ $query = "SELECT FROM "; $query .= $tables[$i]." WHERE "; $qt = "SHOW FIELDS FROM ".$tables[$i]; $fields = JHRquery($qt); for($m = 0; $m < count($fields); $m++){ if(substr($fields[$m]['Type'], 0, 3) != "int" && substr($fields[$m]['Type'], 0, 4) != "date"){ if($m > 0 ){ $query.= " ("; } for( $sc = 0; $sc < count($searchcrit); $sc++){ $query .= $fields[$m]['Field']." LIKE '%".$searchcrit[$sc]."%'"; if($sc != count($searchcrit) - 1){ $query.= " AND "; } } if($m > 0 && $m < count($fields)){ $query.= ") OR "; } } } //this if statement is for if the query ended in WHERE because the only //field types in the table are ints or dates if(substr($query, strlen($query) - 6, 6) != "WHERE " \|\| $tempResult[$tables[$i][0]][0] != ""){ $query = substr($query,0,strlen($query) - 4); //echo $query."<br><br>"; $tempResult[$tables[$i]] = JHRquery($query); } if($tempResult[$tables[$i]][0] != ""){ $realResult[$tables[$i]] = $tempResult[$tables[$i]]; } } return $realResult; } /******************************************************* Function: JHgetnos($where); Purpose: Pares a text file to find out what not to search Author: James R Hartford ******************************************************/ function JHgetnos($where){ //get the words that should not be searched for if($where == "words"){$filename = "no_search_words.txt";} //get the tables that should not be searched else if($where == "tables"){$filename = "no_search_tables.txt";} $handle = fopen ($filename, "r"); $contents = explode(" ", fread ($handle, filesize ($filename))); fclose ($handle); return $contents; } /***************************************************** Function: JHarray_reindex($array); Purpose: reindex an array Author: James R Hartford ******************************************************/ function JHarray_reindex($somearray){ $keys = array_keys($somearray); for($i = 0; $i < count($keys);$i++){ $result[$i] = $somearray[$keys[$i]]; } return $result; } /***************************************************** Function: JHRquery($sql_string); Purpose: Execute query and return results in a 2d array Author: James R Hartford ********************************************************/ function JHRquery ( $query ){ global $dbhost;global $dbname;global $dbuser;global $dbpass; $link = mysql_connect ( $dbhost, $dbuser, $dbpass ); mysql_select_db($dbname) or die("Could not select database $dbhost"); $result = mysql_query($query, $link) or die("Could not execute query $query"); $resultArray = array(); for ($i = 0; $temp = mysql_fetch_array ( $result ); $i++) { if ($temp){$resultArray [$i] = $temp;} } return $resultArray; } ?>
Crawl links on any URL. Just type in the URL and submit... The code extracts all contents from all links on the page and stores them in a text file. Very useful for those considering a fast search mechanism for their site, or those building their own Spider(Crawler)... Please send comments to mynelis@msn.com <? ##------------------------------------------------------- # About the script # ----------------------------------------------- # Code written by Cornelius Duhadzi. # Senior Web Developer - Datawebghana, Ghana. # This script is a spider or crawler that reads all # links on the page provided page. I am developing # this script as part of a bigger search engine # project. Any comment, contribution or collaboration # is welcome. # -------------------------------------------------- # Tel : +233.27.743.78.78 # E-Mail : mynelis@msn.com # -------------------------------------------------- # create a folder called archive in the same # directory. Collection wiil be saved here. ##---------------------------------------------------------- if(!isset($_GET['url'])){ echo "<form action='".$_SERVER['PHP_SELF']."' method='get'>"; echo "URL: <input type='text' name='url' size=40>"; echo "<input type='submit' value='Crawl'>"; } if(isset($_GET['url'])){ class spider{ function crawl($url){ //initialize variables $num_links = 5; //number of links on the page you want to crawl $xdes = ''; $xkey = ''; ini_set('max_execution_time',120); error_reporting(E_ERROR \| E_WARNING \| E_PARSE); //read all links and store in array $server = explode('/',$url); $site = $server[2]; $site = $server[2].'/'.$server[3]; //remove this line before you upload echo "Indexing <b>$site.</b> Please wait...<hr noshade size=1 width=100%>"; flush(); //open and read url if(@fopen($url,'r')){ $follow = file_get_contents($url); $links = ''; //retrieve new links if(@preg_match_all("/(href=)(.?)(>)(.?)(<\/a>)/i",$follow,$href)){ for($a=0;$a<$num_links;$a++){ $link[$a] = $href[2][$a]; //remove all quotes from link $no = array('\'','"'); $yes = array('',''); $link[$a] = str_replace($no,$yes,$link[$a]); //strip out dynamic links if(eregi('\?',$link[$a])){ $link[$a] = explode('?'.$link[$a]); $link[$a] = $link[$a][0]; echo $link[$a]; } //make all urls absolute $use_protocol = array('http://','www'); //modify this line to include any domain name extension(.com,.net,...) for($p=0;$p<count($use_protocol);$p++){ if(eregi($use_protocol[$p],$link[$a])){ $return_url = $link[$a]; } else{ $return_url = "http://$site/$link[$a]"; } } $links .= ",$return_url"; //put all links together separated by commas } } } //open each link, read metatags, titles and content and add to archive $crawl_link = explode(",",$links); if(@fopen($url,"r")){ for($r=0;$r<count($crawl_link);$r++){ if(@fopen($crawl_link[$r],"r")){ echo "Current page: <i>{$crawl_link[$r]}</i>.<br>"; flush(); $page = file_get_contents($crawl_link[$r]); if(@get_meta_tags($crawl_link[$r])){ $meta = get_meta_tags($crawl_link[$r]); $xdes = $meta['description']; $xkey = $meta['keywords']; } //retrieve page title if(preg_match("/<title>(.*?)<\/title>/i",$page,$title)){ $title = $title[1]; } //retrieve page contents, strip html, javascript, whitespace, etc $body = strip_tags($page); //save collection $DL = ":\|:"; $all = $site.$DL.$crawl_link[$r].$DL.$xdes.$DL.$xkey.$DL.$title.$DL.$body; $ext = ".txt"; $dir = "archive/"; $path = $dir.md5($crawl_link[$r]).$ext; $fd = fopen($path,"w+"); fwrite($fd,$all); fclose($fd); } } echo "<hr noshade size=1 width=100%><b>$site</b> indexed successfully.<p>"; echo "<a href='".$_SERVER['PHP_SELF']."'>Index another</a><br>"; } else{ die("Unable to locate <b>$url</b><p><a href='".$_SERVER['PHP_SELF']."'>Index another</a><br>"); } $this->url = $url; $this->xdes = $xdes; $this->xkey = $xkey; $this->title = $title; $this->links = $links; $this->body = $page; } } //calling the function $arch = new spider; $arch->crawl($_GET['url']); } ?>
I needed a way of organizing information in a mySQL db. I decided that I would have a column called "parent" which would have in it the ID number of the "parent" of this entry. Meaning that if the ID of "table" were 1, then the parent of "leg" would be set to 1. At the time I did not know it, but this is called a tree. Now I needed to be able to fetch the information OUT of the database. I wanted it to show in a select box, parent[indent]child etc....<p>so, with MUCH help of Michal Baumann of Maryland/Florida, I (she) wrote this (be sure to change it around as much as your heart/program desires/needs): // un tree so to speak this data tree function untree($parent, $level) { $parentsql = mysql_query("select id, domain from domains where underid=$parent order by domain"); // if it is a leaf (no data underneath it) then return if (!mysql_num_rows($parentsql)) { return; } //else echo the result, and recurse the function (so to speak) else { while($branch = mysql_fetch_row($parentsql)) { $echo_this = "<option value=$branch[0]>"; //give it some indents in the select box for ($x=1; $x<=$level; $x++) { $echo_this .= "   "; } $echo_this.="$branch[1]</option>"; //echo the <option> tag echo $echo_this; //now run this function and find everthing where the parent is equal to the ID of this entry $rename_level = $level; untree($branch[0], ++$rename_level); } } } /* AND HERE IS HOW YOU CALL THE FUNCTION / // get ALL the entries that have NO parent $compsql = mysql_query("select from domains where underid=0 order by domain"); while ($companys = mysql_fetch_array($compsql)) { echo "<option value=$companys[id]>$companys[domain]</option>"; //call the untree function and find all the entries that hive THIS entry as a parent (therefore those entries are children of this one) untree($companys[id], 1); }

php.ComLanka.com PHP Tutorial ,PHP source code, PHP online example