With the advent and dissemination of the internet, the dams of information had been freed; the world’s largest and most updated public library had been founded. And its foundations were heavy with a plethora of knowledge, more than one could have ever imagined in the past!
With the rise of interconnected networks and increased interest of the general public in this new technology in the mid to late 1980’s, that accumulation of knowledge soon spread and grew, meaning that this online library had its first patrons.
All that was missing were the librarians.
Search Engines – The Librarians of the Internet
A search engine is, as defined by Business Dictionary, a ‘computer program that searches databases and internet sites for the documents containing keywords specified by a user.’
This definition while accurate, doesn’t hold water when we look at the term from a broader perspective. They search the vast pages of the internet and return relevant information, but it is this relevancy that the aforementioned definition fails to expound on.
Today’s modern search engines are more than basic crawlers. Like real librarians, they do not merely fetch information as requested. Modern search engines provide suggestions, they auto-complete sentences, and they even assist with that ‘tip-of-the-tongue’ sensation where the user isn’t quite sure of what they want to look for, but will know it when they see it!
Today’s search engines store and remember searches. Their workings have been so discreet that the technology behind it all is hardly questioned, less so marveled. But there is a great deal of technology behind these engines, more so than simple indexing and retrieving. The most prominent of them being machine learning. In fact, Google has even published their own widely discussed courses and documentation on machine learning. And it is machine learning that has shaped our experiences with modern search engines the most.
It is with machine learning that we have search engine optimization (SEO).
Machine Learning – The Groundwork for SEO
Machine learning then is the field in computer science that allows ‘computers the ability to learn without being explicitly programmed to do so.’
This goal of learning, or achieving tasks without human guidance, is a form of artificial intelligence.
With regards to our everyday search engines, machine learning allows for the search engine to auto complete sentences, fetch recommended results, and provide a more detailed and specific range of search results than it would if it were searching by keywords alone.
These methods of choosing data, of creating a more detailed criteria for searching than what the end user entered, is built on the foundations of machine learning.
Its workings are simple to understand, though in reality its complexity to implement has made utilizing it an ongoing process.
While there are many different fields of machine learning implemented in modern search engines, its goals can be broken down into three main categories.
Supervised Learning
This is the process whereby a program is given an input and the relevant ‘correct’ outputs by a human.
From here, the program attempts to identify the rules or patterns that determine what sort of output a desired input should have!
For example, think of searching for a popular celebrity and the search engine automatically ‘knows’ you want to read about that particular person rather than other people with that same name.
Through foresight and design, the search engine has identified that the input of that particular name is associated with that specific person.
Unsupervised Learning
This is similar to the first point, the difference being that now the program itself is trying to discern the correct pattern that connects the inputs and outputs.
Here, data is not necessarily relevant to the user, but its findings often assist the search engine to build its own database and provide feedback to its handlers and programmers.
Such data is less specific and although it does connect the user with an output, it does so without any sort of reinforcement from humans, but on its own accord!
Such programming is also called Hebbian Learning.
Reinforcement Learning
Here the program acts in a dynamic world (stocks) and must achieve a goal within its parameters.
Through trial and error it learns of the rewards and punishments of the particular task at hand changing its strategies to compensate. While more geared towards game development, there are strong implications for our relatively simple search environment.
Imagine our daily searches as a game, with the ‘rewards’ being fast search times and efficient clicks (meaning that the user finds their information easily, without any problems) and the ‘punishments’ being less user engagement and much time lost to find any sort of relevant information.
Setting such parameters and there being (almost) unlimited users to cull data from, the search engine can optimize its search strategy to ensure it maximizes its ‘rewards’ and avoids ‘punishment’.
See: Artificial Intelligence: A Modern Approach (2003), Russel Stuart, Norvig Peter
As mentioned before, these basic tenants of machine learning are easier said than implemented. But it is because of these rules that our search engines do more than simply fetch keyword results. With millions upon millions of page data available, machine learning is the biggest reason why search engines are useful at all!
Such usefulness and the implementation of machine learning is called search engine optimization.
Search Engine Optimization (SEO) – Machine Learning, Applied
With a much better understanding of machine language, we can now confidently jump into search engine optimization.
It is the ‘process of affecting the visibility of a website or a web page in a web search engine’s unpaid results—often referred to as “natural”, “organic”, or “earned” results.’ Put simply, it is the process of making search results smarter.
Instead of displaying undesired results based on phrases and keywords, SEO allows for a more human experience. And this experience of course is forged in the furnace of machine learning. So necessary is machine learning, that you cannot have SEO without it.
In today’s modern search engines, the results we take for granted are more the backbones of that complex learning process, than it is of having rigid criteria.
Perhaps in the past, when search engines first began to emerge, SEO could have been said to be the criteria for which a search engine further narrows down its results after accepting user input.
As of today, it has grown so complex that SEO itself has become machine learning with a purpose. And it is this growth and increased complexity that has fostered this worldwide library.
With the growth of information, the librarians that have emerged are unlike their human counterparts.
Though made by man, these digital bookkeepers, the search engines as we know them, go far beyond and surpass the capacity of any human being. Sifting through the data and weeding the noise, it is through them that the internet has become useful.
And behind it all is the process of becoming smarter by acquiring knowledge. The process of machine learning.