A B H I N AV G U P TA ( 9 9 1 0 1 0 3 4 1 3 )

N I T I S H PA R I K H ( 9 9 1 0 1 0 3 4 0 7 )

R I S H A B H S I N G H ( 9 9 1 0 1 0 3 5 4 4 )

Web Crawler with Email Extractor and Image Extractor

Web Crawler

Web Crawler is a program that, given one or more seed URLs, downloads the web pages associated with these URLs, extracts any hyperlinks contained in them, and recursively continues to download the web pages identified by these hyperlinks. Web crawlers are an important component of web search engines, where they are used to collect the corpus of web pages indexed by the search engine

Web Crawler gives the list of links where the specific word is present in a particular Website and its pages. A Web crawler is an Internet bot that systematically browses the World Wide Wide, typically for the purpose of Web indexing. A Web crawler may also be called a Web spider, an ant, an automatic indexer.

How Web Crawler Works ?

A Web crawler starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.

Email Extractor

Email extracting is the process of obtaining lists of email addresses using various methods for use in bulk email or other. You may need to harvest email addresses when you are conducting a marketing campaign, or when you want to find out something, or send an email to a massive, but targeted, audience. This program is a spider that will detect emails in web sites, through search engines, or just from a file saved on your computer. 

How Email Extractor Works ?

Software Used

Eclipse:In computer programming, Eclipse is a multi-language Integrated development environment (IDE) comprising a base workspace and an extensible plug-in system for customizing the environment. It is written mostly in Java. It can be used to develop applications in Java and, by means of various plug-ins, other programming languages including C, C++, JavaScript, PHP, Python. Development environments include the Eclipse Java development tools (JDT) for Java, Eclipse CDT for C/C++ and Eclipse PDT for PHP, among others.

Screenshots

Image Extractor

Interest in the potential of digital images has increased enormously over the last few years, fuelled at least in part by the rapid growth of imaging on the World-Wide Web. Users in many professional fields are exploiting the opportunities offered by the ability to access and manipulate remotely-stored images in all kinds of new and exciting ways. However, they are also discovering that the process of locating a desired image in a large and varied collection can be a source of considerable .

frustration. The problems of image retrieval are becoming widely recognized, and the search for solutions an increasingly active area for research and development.

PROBLEM STATEMENT

Since the last decade, Features-Based Interactive Image Retrieval was a hot topic research. The computational complexity and the retrieval accuracy are the main problems that FBIIR systems have to avoid.

The aim of this project is to research and implement the potential for using Features-based Image Retrieval methods for querying large-scale image databases. More specifically, the project seeks to identify image features that serve as accurate, yet low dimensional compact, descriptors. In extension it should find methods that have general good retrieval performance that are well suited for scaling. That means that they must be efficient not only in terms of query time but also extraction complexity and storage demands.

OVERALL ARCHITECTURE WITH COMPONENT DESCRIPTION ARCHITECTURAL STRATEGIES

Color Histogram

Color is the most widely used feature because it is the intuitive feature compared with other features and easy to extract from image. However, CBIR system based on color feature often result in disappointment, because it uses global color feature which cannot capture color distributions or textures within the image sometimes. To improve the preferment of the color extraction FBIIRS divides color histogram feature into global and local color extraction. Local color histogram can give some sort of spatial information, however the cons with that it use very large feature vectors.

Geometric Moments

This feature use only one value for the feature vector, however, the performance of current implementation isn’t well scaled, [2] which means when the image size become large, it takes very long time to computer the feature vector. The pros of using this feature combine with other features such co-occurrence, which can provide a better result to user.

Average RGB

The objective of using this feature is to filter out images with larger distance at first stage when multiple feature queries involves. Another reason of choosing this feature, because it uses a small number data to represents the feature vector and it also use less computation compare to others. However, the accuracies of query result could be significantly impact if this feature isn’t combined with other features.

Color Moments

This feature has very reasonable size of feature vector, and the computation isn’t expensive, [4] Colour Moments are measures that can be differentiate images based on their feature of colour, however, the basic of colour moments lays in the assumption that the distribution of colour in an image can be interpreted as a probability distribution. On pros of it is its skewness can be used to measure of the degree of asymmetry in the distribution.

Persistence Module

This module (component) takes care the transaction and persistent of the image features with database. It provides a clear-cut programming interface to other components. Consequently, other module in the system will effortlessly deal with database (such as Feature Extraction and Query module).

FeatureInfo Id Feature name file path vector

Image Represenation in Java

Requirements

Software Items

Window 7/8/8.1 Stability Mac Stability Java Java Runtime Environment & Development Kit Netbeans

Hardware Items

Colored Screen Good Screen Resolution

ScreenShots

ScreenShots

ScreenShots

LIMITATION OF THE SOLUTION

As the results we see that -:„h System is not capable of searching the colored

image on the bases of the sketch of that image.„h If the database is very large (like lacs of images)

then it will take lot of time in extracting features of each and every image.

„h System sometimes hang due to loss of connection to database.

„h If single algorithm is used instead of multiple algorithms the accuracy will come out to be poor.

FINDINGS

1.Index more efficient This system index 1000 sample images in 5 minutes whereas other systems

like QBIC almost took 10 minutes for indexing same number of images. 2. Statable This system more statable as compared to other existing systems. 3. Reusable

Compare with other systems, they provide limited sample image, query from limited image database, but this system can query any sample image, can index any image folder, more reusable

4. Compare with other systems, this provides more searching features. 5. Feedback query

This system provides User feedback Query, user can research from result, increase the accuracy.

CONCLUSION

The extent to which FBIR technology is currently in routine use is clearly still very limited. In particular, FBIR technology has so far had little impact on the more general applications of image searching, such as journalism or home entertainment. Only in very specialist areas such as crime prevention has FBIR technology been adopted to any significant extent. This is no coincidence – while the problems of image retrieval in a general context have not yet been satisfactorily solved, the well-known artificial intelligence principle of exploiting natural constraints has been successfully adopted by system designers working within restricted domains where shape, color or texture features play an important part in retrieval. FBIR at present is still very much a research topic. The technology is exciting but immature, and few operational image archives have yet shown any serious interest in adoption. The crucial question that this report attempts to answer is whether FBIR will turn out to be a flash in the pan, or the wave of the future. It is not as effective as some of its more ardent enthusiasts claim – but it is a lot better than many of its critics allow, and its capabilities are improving all the time. Most current keyword-based image retrieval systems leave a great deal to be desired.

FUTURE WORK

The success of proved both that image retrieval application can be implemented in Java programming language with high performance and Feature-based image retrieval could be a feasible technology in the future. Nevertheless, the project is at basic level thus, many great images retrieval techniques hasn’t implemented, yet. Here is a list of area that can be improved in the future.

Adopting a better cache technique for result image caching, so that the latency of display images will be minimized, as well as using lesser computation and resources.

Implementing a superior ranking algorithm for result image ranking

Getting more visual features extraction module (for example, BEMD filtering for Sketch Detection)

Thank You !Submitted by: Abhinav Gupta 9910103414Nitish Parikh 9910103407Rishabh Singh 9910103544

B.Tech, Cse, 4th yearJIIT-128