API stands for Application Program Interface. In it's simplest definition, an API is a means by which one program can exchange information with another program via it's API. Typically, an application will return some form of XML that another program can read through it's API.
With respect to search results and SEO, there are two APIs currently provided by the two major search engines, Google and Yahoo!. They allow a programmer to extract the search results, and display them as they see fit. The APIs were provided first by Google and then more recently by Yahoo! in an effort to generate interest in their search engines, as well as cut down on the amount of scraping being done by automated programs on the main search sites.
To use an API, a developer is usually required to go through a registration process. Once approved, they can then start using the search results as they want.
Once you've obtained an API key, you can start querying the Google API. Google limits it's API users to 1,000 queries per day per API Key. In other words, if you want to provide a public tool which uses Google's API, it's recommended you require your users to get their own API key, otherwise, within a few short visits, your tool will hit it's 1,000 query limit and stop working the rest of the day.
For some neat things you can do with the Google API, checkout the book Google Hack's (http://www.oreilly.com/catalog/googlehks2/index.html) by Tara Calishain and Rael Dornfest.