PHP

PHP tip: How to convert a relative URL to an absolute URL

Dave_Nadeau — Tue, 27 May 2008 21:39:08 -0700

An absolute URL is complete and ready to use to download a web file. But web pages often include incomplete relative URLs with missing parts, such as an "http" or host name, or the first part of a file path. These parts need to be filled in by copying them from a base absolute URL. This article shows how and includes code to do it.

PHP tip: How to decode HTML entities on a web page

Dave_Nadeau — Sat, 30 Jun 2007 07:27:46 -0700

HTML entities encode special characters and symbols, such as € for €, or © for ©. When building a PHP search engine or web page analysis tool, HTML entities within a page must be decoded into single characters to get clean parsable text. PHP’s standard html_entity_decode() function will do the job, but you must use a rich character encoding, such as UTF-8, and multibyte character strings. This tip shows how.

PHP tip: How to extract keywords from a web page

Dave_Nadeau — Sun, 13 Apr 2008 10:12:00 -0700

Web page keywords characterize the page's topic for a search engine. Extracting keywords requires that you recognize the page's character encoding, strip away HTML tags, scripts, and styles, decode HTML entities, and remove unwanted punctuation, symbols, numbers, and stop words. This article shows how.

PHP tip: How to extract URLs from a CSS file

Dave_Nadeau — Thu, 03 Jan 2008 11:12:49 -0800

Though HTML is usually the focus for extracting URLs for a link checker or analysis tool, CSS files also include URLs. The CSS @import rule uses a URL to include another CSS file, and many style properties include a URL to load an image or other content. This tip shows how to scan a CSS file and extract its URLs.

PHP tip: How to extract URLs from a web page

Dave_Nadeau — Thu, 03 Jan 2008 11:20:03 -0800

URL extraction is at the core of link checkers, search engine spiders, and a variety of web page analysis tools. While <a> and <img> elements are primary sources of URLs, there are more than 70 element attributes with URLs in HTML, XHTML, WML, and assorted HTML extensions. This tip shows how to extract URLs from all of these.

PHP tip: How to get a web page content type

Dave_Nadeau — Sat, 16 Jun 2007 07:03:06 -0700

A web page’s content type tells you the page's MIME type (such as “text/html” or “image/png”) and the character set used by page text. You'll need the character set to interpret the page's characters for text processing for a search engine or keyword extractor. The content type should be in the web server’s HTTP header for the page, but it also can be set in an HTML file’s <meta> tag, or an XML file’s <?xml> tag. This tip shows how to get the page's content type and extract the MIME type and character set.

PHP tip: How to get a web page using CURL

Dave_Nadeau — Sun, 10 Jun 2007 06:02:40 -0700

The first step when building a PHP search engine, link checker, or keyword extractor is to get the web page from the web server. There are several ways to do this. From PHP 4 onwards, the most flexible way uses PHP’s CURL (Client URL) functions. This tip shows how.

PHP tip: How to get a web page using the fopen wrappers

Dave_Nadeau — Sat, 14 Jul 2007 07:30:43 -0700

PHP’s fopen wrappers enable the standard file functions to read web pages from a web server. A few additional calls are needed to set parameters for a web server request and to get the server’s HTTP response header. This tip shows how.

PHP tip: How to parse and build URLs

Dave_Nadeau — Tue, 27 May 2008 21:35:20 -0700

Splitting apart and rebuilding URLs is essential for link checkers, phishing detectors, spiders, and so on. PHP's standard parse_url( ) function works pretty well to parse simple URLs, but it has problems with complex and relative URLs. Once split apart, there is no standard PHP function to reassemble the URL properly. This article reviews the official syntax of URLs, discusses URL parsing complexities, and provides new PHP functions to split apart a URL and join its parts together again.

PHP tip: How to strip HTML tags, scripts, and styles from a web page

Dave_Nadeau — Sat, 01 Sep 2007 20:04:39 -0700

The HTML tags on a web page must be stripped away to get clean text for a PHP search engine, keyword extractor, or some other page analysis tool. PHP's standard strip_tags( ) function will do part of the job, but you need to strip out styles, scripts, embedded objects, and other unwanted page code first. This tip shows how.