I am starting to think about my final year University project and have the idea of writing a web spider. Would this be a good fit for C++?
The basic premise would be a program that web managers can utilise to monitor all interal and external links within their domain, and report on the health of those links at any given interval.
Is this something that is suited to C++ or more so with C#? Also, would you think this is of sufficient difficulty and challenge for a final year project?
I am fairly comfortable with both C++ and C#, however haven't done much web programming.
Difficult - you write everything in C++ starting with the socket code, writing the HTML parser and all the other bits that go along with it.
Easy - you write everything in C# using an HTML component which provides you with a handy DOM interface to the HTML. The project is reduced to little more than a 1-liner "for each link on page do" loop and a bit of stats gathering.
What do YOU find easy or difficult?
What does your academic institution regard as easy or difficult?
I think that, based on your summary, if I did it in C++ rather than C#, it may raise negative questions from my University why I didn't use the 'right tool for the job' and there is 'no need to re-invent the wheel', etc, etc, they are pretty keen on that. I think with the libraries already in C#, that it sounds a little too easy for a final project and would therefore not provide enough substance for me to discuss the project in the dissertation.
Ah well, I think trying to figure out a final year project is the most difficult thing i've had to do whilst at Uni!
Thanks for the response Salem.
I don't know - it's hard to advise on a project when we know so little about your skill or the demands of your faculty.
Being genuinely unique "never been done before" is pretty hard at the undergraduate level. It's always possible to find some similarity with something that has already been done.
Tough choice - use the right tools and be accused of lack of depth, or be accused of not using the right tools for the job.
It sounds like a nice idea - perhaps expand on it by adding more features
- which domains are "stable" and which are "volatile". Eg, knowing www.crap-news.com links always seem to expire within a month vs. www.betternews.com links that last for years.
- which links get used most (integrate with server / firewall / gateway logs).
Why try to maintain a link on a page no-one ever clicks on anyway?
Thanks Salem , I will give it some thought. I am in the lucky position of just starting a 12 month industrial placement prior to my final year at Uni, so I have a little time to think things through and learn new skills. Ideally I want to start actually coding it within the next 3-4 months, so I will keep reading and searching for ideas and/or expansions to this idea.
Don't know if it'll be useful, but what I did was listen to interviews with prominent people in my areas of interest and see what ideas they had for what's going to happen in the future.
For instance, I got my idea from watching an interview with the lead dev on Microsoft's driver framework when it was first released. The interviewer asked him if managed languages would ever be involved on the kernel level, I can't remember what the response was but that became my project. 14-15 months later, I took my laptop and a Windows XP VM into a meeting with the profs marking my dissertation and demonstrated the Kenco language (named after the coffee, since I was in Java class at the time), the compiler, a runtime and a short proof of concept.
So yeah, I'd say make sure it's a project you'll enjoy because you're going to be seeing quite of lot of it.
Your industrial placement will be a good source for ideas :)
With the added bonus of a job offer if you make a decent attempt.
Makes a change from the usual "gimme project title" types who roll through with 1-post and 24 hour deadlines :)
There's some good advice there. It is difficult trying to find something at undergraduate level that's innovative and progressive. They tend to be more so at masters or PHD level.
Originally Posted by adeyblue
Thanks for the advice.
Originally Posted by Salem
As a mature student (34 upon graduation), I feel that I need to make my CV stand out more than most. One way of doing that is having a project on there that catches the eye. Hopefully, will think of one over the next few months.