I am trying to build a service which would search across Google, Twitter, Youtube and other sites to get content relevant to a particular topic and provide it on one screen to my users ( like a meta search engine)
They sure will. Google even has an API for this( https://developers.google.com/custom-search/docs/structured_data#overview ) as does Twitter, YouTube and probably the other sites you are looking into. The major problem you will have is the limits and these services will get pretty expensive fast if you reach them. I would look into something like Apache Solr or Elasticsearch to help you store your results so this is less of an issue. Good luck!
Answered 7 years ago
It sounds like you're trying to build a site that takes a user's query and then goes out and runs it against a number of major search platforms, then aggregates those results.
You might be able to get away with this on a small scale, but these companies put enormous amounts of time and money into making their search experience fast and user-friendly, usually so they can sell ad space against it, or otherwise monetize users' search intent behavior.
For you to take the results they generate and display them in a different experience -- one that they can't monetize, and one that you might be trying to monetize -- would put your interests directly opposed to theirs.
Their first layer of defense would likely be captchas, rate-limiting or other technological impediments which would make it very tough for your site to continue to operate. If you find a way past those, then you'll likely start hearing from their lawyers, a solution that will be even more expensive to engineer around.
Building a website on top of someone else's platform always has an element of risk. But when your interests are so poorly aligned with theirs, then the risk of them cutting you off and shutting you down one way or another is virtually 100%.
Answered 6 years ago