Search Engine with DFS-based Web Crawling

Table of Contents

• Introduction
• Features
• Technology Stack
• Prerequisites
• Installation
• Usage
  • Performing a Search
  • Getting User Search History
• Running the Application Locally
• Contributing

---

SearchEngine.mp4

Introduction

This project is designed to provide a practical understanding of search engines and web crawling. It allows users to perform searches and view their search history. The search functionality is complex, involving various algorithms and techniques.

The core of this project is to retrieve the top 30 search results based on a user's query, simulating the functionality of a real search engine. It also maintains a history of user searches, which can be useful for analysis and personalization.

Features

• Search Engine: Retrieve the top 30 search results for a given query.
• Search History: View a history of your past search queries.
• DFS-based Web Crawling: The engine uses depth limited Depth-First Search (DFS) to navigate and index web pages.
• Technologies: Utilizes HTML, CSS, JSP, Java Servlets, and MySQL for various components.

Technology Stack

The project is built on the following technology stack:

• Frontend:
  • HTML
  • CSS
• Backend:
  • Java Servlets
  • JSP (JavaServer Pages)
• Database:
  • MySQL
• Web Crawling:
  • Jsoup for web scraping

Prerequisites

Before you begin, ensure you have the following:

• Java Development Kit (JDK)
• Apache Tomcat (e.g., version 9.0.82)
• MySQL (e.g., MySQL Server)
• mysql-connector-java (e.g., version 8.1.0)
• Smart Tomcat Plugin
• javax.servlet-api
• jsoup (e.g., version 1.15.4)

Installation

1. Clone this repository to your local machine using below command:

      git clone https://github.com/jangir02vishal/Search-Engine.git
   

2. Create a MySQL database and configure the database connection in src/main/java/com/Accio/DatabaseConnection.java.

3. Build the project.

Usage

Performing a Search

1. Start the Apache Tomcat server.
2. Open a web browser and navigate to http://localhost:8080/SearchEngine/.
3. Enter your search query in the provided search bar.
4. Click the "Search" button to get the top 30 search results based on your query.

Getting User Search History

1. On the search page, click the "History" button.
2. You'll be able to view your search history, including past queries and the date and time of the searches.

Running the Application Locally

To run this application locally:

1. Make sure Apache Tomcat is installed and configured.
2. Deploy the project to Tomcat using your IDE or by manually copying the project folder to the Tomcat webapps directory.
3. Start the Tomcat server.
4. Access the application via your web browser at http://localhost:8080/SearchEngine/.

Contributing

We welcome contributions to improve this project. Feel free to open issues, submit pull requests, or suggest improvements.

Homepage -

Search results-

History results -