Extracting data from internet

Posted by Ankiov Spetsnaz on Programmers See other posts from Programmers or by Ankiov Spetsnaz
Published on 2012-11-13T04:17:57Z Indexed on 2012/11/13 5:14 UTC
Read the original article Hit count: 492

Filed under:

java

|

data-mining

|

web-crawler

I would like to extract data from internet like www.mozenda.com does but I want to write my own program to do that. Specific data I'm looking for is various event data.

Based on my research, I think custom web crawler is my answer but I Would like to confirm the answer and see if there are any suggestion to make custom web crawlers if web crawler indeed is an answer.

Personally, I would prefer Java and I'm planning on using Glassfish technology if that matters...

© Programmers or respective owner

Related posts about java

Tomcat 6: Access Control Exception?

as seen on Server Fault - Search for 'Server Fault'
I'm trying to setup a tomcat6 server, and I'm trying to match another setup someone else established. However, my deployment (default Ubuntu install) uses a policy.d/ directory structure, and the established server just uses a catalina.policy file. I've tried setting every entry in policy.d to match… >>> More
Problem in creation MDB Queue connection at Jboss StartUp

as seen on Stack Overflow - Search for 'Stack Overflow'
I am not able to create a Queue connection in JBOSS4.2.3GA Version & Java1.5, as I am using MDB as per the below details. I am putting this MDB in a jar file(named utsJar.jar) and copied it in deploy folder of JBOSS, In the test env. this MDB works well but in another env. [ env settings and… >>> More
failing to establish connection between Postgres db and gwt

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am using Postgres and gwt 2.0 for one of my applications. I am facing problem connecting to the database. When I try to connect it gives "ClassNotFoundException". Here is what I get when I try to connect to database: java.lang.ClassNotFoundException: org.postgresql.Driver at java.net… >>> More
failing to establish connection between postgre db and gwt

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, For i am using postgre and gwt 2.0 for one of my applications. I am facing problem connecting to the database. When i try to connect it gives "ClassNotFoundException". Here is what i get when i try to connect to database: java.lang.ClassNotFoundException: org.postgresql.Driver at java.net… >>> More
Migration and deployement problems JBoss 4.2.2.GA to JBoss 6.0.0.M2

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I'm trying to migrate an application running on JBoss 4.2.2.GA to JBoss 6.0.0.M2 I give you some log to explain my problem : boot.log : 2010-03-16 09:59:29,406 ERROR [org.jboss.system.server.profileservice.ProfileServiceBootstrap] (Thread-2) Failed to load profile: Summary of incomplete deployments… >>> More

Related posts about data-mining

SQLAuthority News – Links to Book On Line – Data Mining Algorithms (Analysis Services – Data Mining)

as seen on SQL Authority - Search for 'SQL Authority'
I have quite often received request for the Data Mining Algorithms details. Book Online has wonderful resources for the same. I suggest to read them here. Data Mining Algorithms (Analysis Services – Data Mining) The data mining algorithm is the mechanism that creates a data mining model. To… >>> More
Data Mining Resources

as seen on SQL Blog - Search for 'SQL Blog'
There are many different types of analyses, each one with its own pros and cons. Relational reports have a predefined structure, and end users cannot change it. They are simple to use for end users. Reports can use real-time data and snapshots of data to show the state of a report at specific points… >>> More
Integrating Data Mining into your BI Solution (Presentation)

as seen on SQLIS - Search for 'SQLIS'
I recently gave a live meeting presentation to the UK User Group on Integrating Data Mining into your BI Solution. In it I talk about and demo ways of using your data mining models inside Integration Services, Analysis Services and Reporting Services. This is the first in a series of presentations… >>> More
What data mining tools do you use?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello everyone, Besides the two well-known Open Source tools RapidMiner and Weka, are there any other good tools (either Open Source or Commercial), which you can recommend for data mining? Thanks in advance! >>> More
NEW 2-Day Instructor Led Course on Oracle Data Mining Now Available!

as seen on Oracle Blogs - Search for 'Oracle Blogs'
A NEW 2-Day Instructor Led Course on Oracle Data Mining has been developed for customers and anyone wanting to learn more about data mining, predictive analytics and knowledge discovery inside the Oracle Database. Course Objectives: Explain basic data mining concepts and… >>> More