What's New

New updates and improvements to SearchApi

December 2024: Update

New
Here’s what’s new at SearchApi, along with a quick peek at what’s coming next!

✈️ ​Google Flights API​

We’re excited to introduce the ​Google Flights API​. It delivers real-time flight data, including:
  • Best flight options
  • Price insights
  • Airport details
  • Departure and arrival times
  • Flight duration
  • Airline information
  • In-flight amenities
  • Carbon emission data ...and more!

⚡ Amazon API Performance


In November, we revamped our internal monitoring system to analyze usage patterns across engines. This allowed us to minimize spikes and stabilize the Amazon Search API. While improvements are ongoing, we’re committed to optimizing all engines further.

Amazon Search API Performance (2024 Nov - 2024 - December)
Furthermore, we’re confident about our upcoming Enhanced Speed plans. After successful internal testing with select clients, we’re rolling these out publicly next month.

🛒 ​Google Shopping API​ (November changes)


Google has officially removed support for the old layout in specific regions:
  • United States: gl=us
  • Canada: gl=ca
  • New Zealand: gl=nz
  • Australia: gl=au
  • Japan: gl=jp ...and several small territories (e.g., Norfolk Island: gl=nf).
Other regions continue to support the regular layout.

Parameter Updates:
  • ​price_min​ and ​price_max​ are now strictly applied in the new layout.
  • When using the shoprs parameter:
    1. Start with price_min and price_max filters.
    2. Use shoprs with additional filters from the JSON response. This ensures price filters are applied before further refinements.

🔧 Engine Improvements:


Google Search API

  • Fixed inline_videos missing keys: link, length, source, channel.
  • Fixed inline_tweets selectors.
  • Fixed ads.phone and ads.thumbnail selectors.
  • Fixed knowledge_graph.website and service_options selectors.

Bing Search API

  • Fixed organic_results.sitelinks selectors.

DuckDuckGo API

  • Fixed an issue causing server errors with the knowledge_graph key.

Google Maps Reviews API

  • Fixed reviews.user selectors.

Google Shopping API

  • Extracted tbn images instead of base64 for shopping_results.
  • Fixed price_min and price_max request parameters.

Amazon Search API

  • Improved filters.text clarity (e.g., “Top Brands” filter).
  • Decoded tracking links in organic_results.

Google Images API

  • Fixed the parser for the new layout (keys remain unchanged).
  • Fixed usage_rights request parameter values.

YouTube Search API


Other


⏭️ What's coming next?


  • Yahoo Search API
  • Enhanced Speed plans ⚡
  • Google AI Overview
  • Google Hotels API
  • Google Reverse Image API
  • Yandex Search API
  • Base64 image hosting on our side. This will shorten the LLM context if you want to use raw JSON responses.

Stay tuned for more updates. As always, we're committed to providing you with the best experience.

November 2024: Update

New
We're thrilled to update you with two new engines and a range of fixes to existing engines. Here’s a detailed look at the latest features and improvements and a glimpse of what’s coming next.

​DuckDuckGo API​



DuckDuckGo API is a new addition to the Google, Bing, and Baidu SERP APIs. This API delivers data such as organic results, ads, knowledge graphs, AI overviews, answer boxes, and many more. We kept the JSON response keys consistent with the SERP engines to make integration easier.

​Apple App Store Reviews API​



Our new Apple App Store Reviews API works alongside the Google Play Store Reviews API. It provides review ID, title, text, rating, and additional review data. JSON keys are consistent with Google Play for straightforward integration.

🔧 Engine Improvements


Google Search API

  • Updated `domains`, `gl`, and `hl` constants list.
  • Adjusted `inline_images` selectors.
  • Fixed `open_hours` in `knowledge_graph` and `answer_box`.
  • Fixed `ads.sitelinks.expanded` selectors.
  • Fixed `inline_videos.key_moments` selectors.
  • Resolved rare issues when empty results were returned for valid queries.

Google Lens API

  • Resolved issues where invalid URL characters were causing some searches to fail.

Google Maps API

  • Added a new key - `place_result.located_in`.

Google Videos API

  • Fixed an issue where tablet and mobile requests were failing.

Google Images API

  • Fixed pagination issue.

Google Shopping API

  • Fixed an issue where collapsed filters were not extracted.
  • Resolved issues where searches in Japan & Canada were failing.

Amazon Search API

  • Fixed an issue where `variants` were not parsed correctly.

Amazon Product API

  • Fixed an issue where product specifications were not parsed correctly.

YouTube API

  • Fixed an issue where empty `hl` and `gl` values failed.

YouTube Transcripts API

  • Added support for video links inside `video_id` parameter, for example - `video_id=https://www.youtube.com/watch?v=RcYjXbSJBN8`. We automatically extract the `video_id` now.
  • Added a new parameter - `​transcript_type,` that defines the transcript preference when both auto-generated and manually uploaded transcripts are available.

YouTube Comments API

  • Added `search_information`.
  • Fixed `error` message if comments are disabled.

Other

  • Shortened API examples in Google documentation.
  • Added `subscription` details to Account API response (GET `/api/v1/me?api_key=API_KEY`).

🚀 Performance Improvements:


We’ve upgraded our database to a larger instance, leveraging AWS Graviton3 processors and upgrading PostgreSQL and Redis to the latest versions. This upgrade, which required a brief 3-minute downtime on October 3rd, scales our search capacity fourfold. In the future, we may explore RDS Blue/Green deployments to minimize downtime further. The result? Faster operations across API and Web.

Next month, we’ll roll out 2x performance plans designed for high-demand LLM use cases. Thanks to ongoing optimizations and expanded proxy partnerships, these plans greatly boost speed and reliability.

⏭️ What's coming next?


  • Google AI Overview
  • Google Flights API
  • Google Hotels API
  • Google Reverse Image API
  • Yandex Search API
  • Yahoo Search API
  • 2x Performance plans
  • Base64 image hosting on our side. This will shorten the LLM context if you want to use raw JSON responses.

Stay tuned for more updates. As always, we're committed to providing you with the best experience.

October 2024: Update

New
This month, we’ve rolled out four new engines, an interactive playground, and a range of minor fixes. Here’s a detailed look at the latest features and improvements and a sneak peek of what’s coming next.

​Shein Search API​

Shein Search API is a new addition to the Google Shopping and Amazon Search APIs. Shein Search API provides product IDs, prices, titles, links, and more data. To make integration easier, we kept the JSON response keys as consistent as possible with the Amazon Search API.

​Shein Product API​

Shein Product API offers detailed product information such as specifications, stock availability, and variants. This API makes retrieving real-time data on product options like sizes or colors easy.

​Apple App Store Details API​

Apple Product API provides detailed app information from the Apple App Store. This API delivers data such as app descriptions, ratings, version history, supported devices, screenshots, and developer details.

​Apple App Store Top Charts API​

Apple App Store Top Charts API provides real-time access to the most popular apps on the Apple App Store. With this API, you can retrieve app rankings, titles, genres, developer details, ratings, and review breakdowns across 155 countries.

🚀 Interactive Playground


We’re excited to announce the launch of our first interactive playground on our marketing pages! This is a sneak peek of what's coming to our documentation in the future. It allows API testing directly in the browser without opening a new tab.

🔧 Engine Improvements:


Google Search API

  • Fixed a case when searching for `weather_results` triggered an error.

Google News API

  • Added `sort_by` request parameter.

Amazon Search API

  • Added `search_information`, including displayed query, max, and min prices.

Google Jobs API

  • Fixed `extensions` and `detected_extensions` extraction.
  • Fixed case when `description` wasn't returned.
  • Fixed issues parsing the last page when using pagination.

Google Shopping API

  • Improved parsed thumbnail quality.
  • Fixed case when the thumbnail was not parsed correctly.

YouTube API

  • Fixed `shorts` extraction.

YouTube Channel API

  • Fixed `shorts` extraction.

YouTube Trends API

  • Fixed `shorts` extraction.

YouTube Video API

  • Fixed `key_moments` extraction.
  • Fixed `chapters` extraction.

Other

  • Added ​Career page​.
  • Fixed invalid encoding sequence for customer-passed user agents.

🔗 New Integrations


  • ​langchain4j​ - A Java version of LangChain, allowing smooth language model workflows in Java applications.
  • ​open-webui​ - A simple, user-friendly WebUI for interacting with AI models, formerly known as Ollama WebUI.
  • ​anything-llm​ - An all-in-one AI tool for desktop and Docker, with built-in RAG, AI agents, and other useful features.
  • ​openagi​ - An open-source project that builds open agents and AGI that everyone can use.
  • ​gpt-researcher​ - An LLM-based research tool that automatically searches and gathers information on any topic with little user input.

⏭️ What's coming next?


  • Google AI Overviews
  • Google Flights API
  • Google Hotels API
  • Apple App Reviews API
  • Yandex Search API
  • DuckDuckGo Search API
  • Google Reverse Image API
  • Open-source integrations
  • Search History API

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

September 2024: Update

New
We've pushed some significant updates to our APIs and parsers this month. Here's your changelog of new features, critical fixes, and a roadmap of what we're building next.

​New Google Shopping Layout​


Important: The new Google Shopping layout is opt-in. You will only be affected by these changes if you explicitly pass these gl values: `us`, `au`, `ca`, `tk`, `nf`, `nz`, `hm`, `cx`, `cc` and `hl=en` as request parameters. If you do not use these parameters, the old layout will continue to be used by default.

Now, let’s explore the key changes introduced in the new Google Shopping layout:
Changes to Request Parameters

  • [BREAKING] The `tbs` request parameter is no longer supported in the new layout. Instead of `tbs`, we introduce the new `shoprs` parameter that is responsible for custom filters. You can also apply filters in the search query directly.
  • [BREAKING] The `sort_by` request parameter is no longer supported in the new layout.
  • [BREAKING] The `num` request parameter is no longer supported in the new layout.
  • [BREAKING] The `price_min`, `price_max`, and `condition` request parameters are still supported; however, their functionality is different. Many filters can be triggered in the `q` search query parameter. That's exactly how we support these 3 filters: by adding `under`, `over`, `new`, or `used` keywords in the search query.
Remember, many of these filters can now be applied directly in the search query for greater flexibility. Here are some examples of how you can use it:

  • Search by price: charizard over $300.
  • Search by size: mens size 8.5 nike air.
  • Search by color: Playstation white.
  • Search nearby stores: Apparel nearby.
  • Search by condition: iphone used, macbook new.
  • Search by brand: prada dress.
  • Search discounted products: headphones on sale.

Updates to Parsed JSON Fields

  • [BREAKING] The `featured_results` top key removed, replaced by `popular_products`.
  • [BREAKING] Removed top keys: `related_results`, `nearby_results`, `categories`, `related_searches`, and `pagination`.
  • [BREAKING] `filters.options.tbs` changed to`filters.options.shoprs`.
  • [BREAKING] The `shopping_results.link` key is no longer supported. Use Google Product API to get direct shopping links.
  • [BREAKING] `shopping_results.extensions` and `shopping_results.merchant` keys are no longer supported.
  • [BREAKING] `shopping_ads.durability` and `shopping_results.durability` renamed to `condition`.
  • [NEW] Added keys: `shopping_results.delivery_return` and `shopping_results.snippet`.
  • [Planned] Direct product links.

Other keys stay the same as before.

How to Access the Old Shopping Layout

Omit passing the `hl` parameter.

​Old Google Shopping Layout (No changes in all other regions)


It's important to note that the old Google Shopping layout is still functional in other regions. The only case when this layout is not being triggered is when setting `gl` values to `us`, `au`, `ca`, `tk`, `nf`, `nz`, `hm`, `cx`, `cc` and `hl=en` request parameters. Any other combination of these parameters will trigger an old layout. For instance, setting `gl=de` and `hl=de` (or `gl=us` and `hl=es`) will allow you to view the old layout, which remains unaffected by the recent changes implemented in the new layout.

🌟 New Engine - Apple App Store API​


We're excited to release the Apple App Store API, a new addition to the Google Play Store API. Apple App Store API provides product id, rating, reviews count, supported devices, languages, and more data. We are going to add App Details API, App Store Reviews API, and App Store Top Charts API next.

🏎️ Request time taken (20%-30% across engines)

We improved `request_time_taken` by 20%-30% across all the engines. It is still early, and we are heavily experimenting, but we know we can offer exceptional speed in the market. We will announce dedicated ultra-high-speed plans later this year.

🔧 Engine Improvements


Google Search API

  • Fixed `inline_tweets` selectors.
  • Fixed `search_information.detected_location`.
  • Fixed `salary_estimates` selectors.
  • Fixed `from_sources_across_the_web` selectors.
  • Renamed `job_results` to `jobs` and made structure identical to Google Jobs API.
  • Added support for a new `organic_results.sitelinks` case.
  • Fixed `knowledge_graph` for shopping entity.
  • Fixed `knowledge_graph.type` as sometimes it would return a rating instead of type.
  • Fixed `things_to_do` selectors.
  • Added a new `answer_box` type - `core_answers`.
  • Fixed `related_questions` case when an empty field was being returned.

Bing Search API

  • Fixed `organic_results.cached_page_link`.
  • Fixed `inline_images` thumbnails. Now we return higher resolution images.

Google Lens API

  • Performance improvements.

Google Play Store API

  • Fixed `highlights.id` and `highlights.link` selectors.

Google Play Product API

  • Fixed `product.rating` and `product.reviews` selectors.
  • Fixed a case when reviews were not returned.
  • Fixed a case when `similar_products` were not returned.

Google Trends Autocomplete API

  • Fixed an issue that was triggering timeouts.

Other

  • Adjusted website colors to improve accessibility.
  • Updated the look of API Keys show page
  • We now run Rails 7.2 with YJIT enabled in production.

⏭️ What's coming next?


  • Google Flights API
  • Google Hotels API
  • Apple App Reviews API
  • Apple App Product API
  • Shein Search API
  • Shein Product API
  • Yandex Search API
  • Interactive playground
  • Open-Source integrations
  • More OpenAPI schemas
  • Careers page

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

August 2024: Update

New
We've pushed some significant updates to our APIs and parsers this month. Here's your changelog of new features, critical fixes, and a roadmap of what we're building next. Let's dive into the technical nits.

🌟 Google Jobs New UI


Google Jobs changed the layout globally and we already released an updated parser. It comes with infinite pagination support.


These are the newest updates to the new Google Jobs API:

Pagination Changes

  • [BREAKING] The `num` and `page` request parameters are no longer supported. To retrieve the next page of results, use the `next_page_token` from `pagination.next_page_token`.

Parsed JSON Updates

  • [NEW] Multiple `apply_links` are available in a new Google Jobs UI.
  • [BREAKING] `company_website`, and `company_web_results_link` are not present anymore. We would need to implement a separate API to retrieve this information.
  • As before, `via`, `description`, `location`, `job_highlights`, `sharing_link`, `thumbnail`, and `detected_extensions` with full `extensions` coverage is being parsed.

Request Parameters

  • [BREAKING] Old filters such as `chips`, `lrad`, `ltype` were removed. All filters can be used in text searches.
  • `gl`, `hl`, `google_domain`, and `location/uule` have less influence and all filters can be applied by text inside the `q` parameter. (For example - Engineer in California).
  • Search by location: software engineer jobs in San Francisco.
  • Search by major or field of study: biology major jobs.
  • Search from a specific website: jobs from LinkedIn.
  • Search by military occupational specialty (MOS) code: veteran jobs with MOS 11B.
  • Search for remote jobs: remote customer service jobs.

🔧 Engine Improvements


Google Search API

  • Added new data inside the `offers` key in `knowledge_graph`.
  • Fixed the `related_question` answer as in some cases it was not parsed.
  • Fixed `inline_recipes` parser.
  • Fixed `answer_box` with type - `location_type`.
  • New parsed field - `popular_destinations`.
  • New parsed field - `universities`.
  • Fixed number extraction in finance type answer box.

Google Jobs API

  • Updated Google Jobs supported `gl` parameter values.
  • Fixed a case when Google Jobs API wouldn't return any results.

Google Shopping API

  • Updated Google Shopping API examples in the documentation.
  • Fixed a case when Google Shopping would return only `organic_results`.

Amazon Search API

  • Fixed `link` for products that don't have a title.
  • Fixed a case where `specifications` sometimes didn't yield any results.
  • Added `price_min` and `price_max` request parameters.

Google Maps API

  • Fixed a case when using specific coordinates didn't return any results.

Google Maps Place API

  • Updated `extensions` structure to be identical to Google Maps API.

Google Maps Reviews API

  • Implemented support for the `place_id` request parameter. Here is an API example.

Google Trends API

  • Improved Google Trends success rate.

Bing API


Google Scholar API

  • Added a `source` request parameter that works exactly the same as writing a source in a query. Only one source can be used while using the `source` parameter.

Google Patents API

  • Fixed an issue when empty request parameters were used.

Google Videos API

  • Fixed typo `highlighed` and changed to `highlighted` inside `videos` key.
  • Fixed the `shorts` parser on the `device=mobile` request parameter.

Other

  • Added Amazon Pay as a payment option.

⏭️ What are we working on next?


  • OpenAPI schema
  • Documentation improvements
  • Google Flights API
  • Google Hotels API
  • Apple App Store API
  • Yandex Search API
  • CrewAI integration
  • Search anonymization

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

July 2024: Update

New
We’re happy to share last month’s technical achievements, featuring significant parsing speed improvements, a newly released engine, and a handful of minor fixes.

🚀 Parsing Speed Performance




We've drastically enhanced our parsing speed across multiple engines. For instance, parsing a Google search for "Ice cream" with `num=100` has been reduced from 2.29 seconds to 0.5 seconds, and an Amazon product search for ASIN "BOBTD62N4J" has decreased from 0.72 seconds to just 0.07 seconds. That's a whopping 10x improvement for Amazon and almost 5x for Google. 😍

🌟 ​Bing News API​




We're excited to release the ​Bing News API​, providing organic results, trending, and related topics. You can also search by categories like Sports, Local, or Science. It supports mobile, tablet, and desktop layouts. We made organic results JSON keys consistent with the Google News API engine.

🔧 Engine Improvements


Google Trends API

  • Added support for an empty `q` value while searching for Related Topics and Related Queries. Also for Timeseries while searching with category. This enables you to get overall trends in specific categories or regions.

Google Maps Photos API

  • Implemented support for the `place_id` request parameter. ​Here​ is an API example.

Google Search API

  • Fixed `courses` parser. In some cases, this field wasn't parsed correctly.
  • Added `did_you_mean` parser in `search_information`.
  • Improved `snippet` extraction for an edge case.
  • Fixed `shopping_ads` to return all the results.
  • Fixed `weather_results` for `device=mobile` parameter to return all the results.
  • Fixed `expanded_links` for `device=mobile` parameter.
  • Fixed Swiss number extraction.

Google Patents API

  • Fixed case when `frequency` wasn't parsed correctly.

Google Play Product API

  • Fixed case when `category` wasn't parsed correctly.

Baidu Search API

  • Fixed request timeouts for specific queries such as weather information.

Amazon Search API

  • Improved parsing performance by 50%, independent of the overall speed enhancements.

Other

  • Improved rendering of images in HTML across different APIs.
  • Fixed API code example in Go language.

⏭️ What are we working on next?


  • Google Trends API. Getting back to a 100% success rate
  • Search History anonymization
  • Google Flights API
  • Google Hotels API
  • Open source integrations
  • Playground

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

June 2024: Update

New
Today, we're thrilled to update you on our last month's improvements: released a new search engine, team features, published 2 new open-source integrations and added a bunch of minor fixes.

🌟 New Engines


We're excited to release Baidu Search API offering organic results, answer box, knowledge graph, related questions, top searches and more results.

To ease integration, JSON keys align closely with those of the Google Search or Bing Search APIs. You can configure Baidu to return up to 50 organic results per search. Perfect for building tools across.

👥 Team Accounts


To enable team functionality, users must have a Production plan or higher. When a user upgrades to a Production plan or higher, their account is automatically converted to a team account. Users on the Free and Developer plans will see a clear explanation on the Team page about how to access this feature.

Owners and Admins can invite other users by providing their name and email. Invited users can join the team using social sign-up, email/password, or an existing account. Additionally, the Owner can transfer the account to another user. Invited users must verify their email, and verified emails will be displayed on the Team page.

Team admin perspective

Admins have full access to manage team settings, invite and remove users, and monitor team activity. They can also manage billing and account settings, ensuring the team operates smoothly.
Developer perspective

Developers have access to the Search History and Search Analytics but do not have the same administrative privileges as admins. They cannot manage team settings or invite/remove users or manage billing, allowing them to focus on development tasks without altering team configurations.

🔗 Dify Integration​


​Dify - an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features, and more, letting you quickly go from prototype to production.

We are excited to announce that SearchApi now integrates with four engines: Google Search, Google Jobs, Google News, and YouTube Transcripts APIs.

🔧 Engine Improvements


Google Search API:

  • Fixed title and price in `things_to_do` which sometimes wouldn't return a value.
  • Fixed `about_this_results` on `device=mobile` parameter.
  • Fixed `knowledge_graph` order & website links.
  • Fixed `knowledge_graph` offers.

Google Shopping API:

  • Added new key - `extracted_offers` in `shopping_results` and `featured_results`.

Amazon Search API:

  • Added new request parameters - `rh` and `sort_by`.
  • Added new parsing fields: `more_offers`, `attributes`, `fulfillment`, `prices`, and more.

Google Play Product:

  • Fixed pagination. Sometimes it didn't return `next_page_token`.

🔗 Other Integrations:


  • ​chat-ui - Open source codebase powering the HuggingChat app.

⏭️ What are we working next?


  • Playground
  • Bing News API
  • Google Flights API
  • Google Hotels API
  • Open source integrations
  • Performance improvements. We have some significant gains to show 😊
  • Search History anonymization to satisfy strict legal requirements

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

May 2024: Update

New
Today, we're thrilled to update you on our last month's improvements: released 4 new engines and added a bunch of minor fixes.

🌟 Freshly Released Engines:



Our new Google Maps Reviews API lets you access reviews of a given place. You can sort reviews by search query, topic id, or your chosen sorting parameter. API returns detailed information about the place, relevant topics, and the reviews themselves, making it easier to gather targeted insights.

Similar to the Google Maps Reviews API, the YouTube Comments API allows you to retrieve comments and their replies from any specific video.


Another newly released engine - YouTube Trending API. It allows you to access the most trending videos, music, and films based on a selected location, ensuring you stay informed about the most popular content worldwide.


Introducing our latest release – the YouTube Channel API. While this API efficiently scrapes videos, shorts, and other content from YouTube channels, its standout feature is the "About" section. You can easily scrape channel links, description, subscribers count, total views, join date, account location and other key details.

🔧 Engine Improvements:


Google Search API:

  • Fixed ratings & reviews in the `inline_shopping` block.
  • Added new explore brands field to the JSON response.
  • Fixed `discussions_and_forums` extraction.
  • Fixed `is_live` & `is_video` extraction in `top_stories` (both mobile & desktop).

Google News API:

  • Fixed `organic_results` selectors which caused an empty first organic result in some cases.

Google Lens API:

  • Text Results - fixed an issue that caused errors with wrongly identified text results.

Google Shopping API:

  • Fixed the `prds` parameter; it is no longer necessary to include the `pid:` value inside `prds`.
  • Fixed `product_link` inside `organic_results`. Default `gl` is now `us` (previously it was `en`, which caused some links to not open correctly).
  • Fixed `link` in `organic_result`. Corrected an issue where some links from Google Shopping pages resulted in 404 errors due to improper parsing of URL query parameters. We've updated the logic to ensure URLs are correctly formatted and fully accessible.

Google Product Reviews API:

  • Fixed product rating, which sometimes wouldn't return a rating value.
  • Added a new request parameter, "rating," which allows filtering results based on the rating score.

Google Images API:

  • Fixed `tbs` parameter. Some filters, like `size` or `color` were not working due to this issue.
  • Improved documentation examples.

Google Finance API:

  • Fixed an issue in financials where data for specific quarters was not being returned.​
  • Fixed an issue where financials would show infinite values.

YouTube Search API:

  • Fixed an issue that affected `organic_results` result positioning. Autocorrect result block was sometimes identified as the first result. It caused `"position": 1` to be missing in some cases.

Amazon Search API:

  • Fixed GTIN search. Now products that have similar "GTIN" are not selected anymore.

Other improvements:

  • Improved performance of Search History and Search Analytics pages.

💥 Global Google Outage Impact


We experienced disruptions due to a recent global outage of Google Search and its related services. This led to some users encountering a 502 (Bad Gateway) error. The interruptions lasted approximately 20-30 minutes on May 1st and briefly for an additional 5-10 minutes on May 2nd. As a result, SearchApi API calls related to Google searches timed out during these periods causing 503 response code. Please be aware that such events are infrequent and beyond our control.

⏭️ What are we working next?


  • Team Accounts
  • Playground
  • Baidu Search API
  • Amazon Search API improvements
  • Google Hotels API
  • Google Flights API
  • Open source integrations

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

April 2024: Update

New
This month, we released a major addition to our Google Maps API suite: the Google Maps Place API. Alongside this, we added four additional engines, a new search analytics feature, and several engine improvements.

🌟 Freshly Released Engines:


Our new Google Maps Place API lets you access detailed information about specific locations, including names, addresses, ratings, reviews, admissions, nearby popular places, and more!

With the Photos API, it is easy to integrate high-quality images of places into your applications. This API provides access to millions of photos, helping you create more visually engaging user experiences.


This API offers insights into the top trending topics and queries on Google Trends, updated daily. It's perfect for keeping your content relevant and engaging.

Get real-time access to the hottest trends with Google Trends Trending Realtime API. This API provides instant updates on the latest trending topics and queries, allowing you to react quickly to emerging trends and capture your audience's attention.

Access real-time price changes, historical data, quarterly and annual company financials, exchange rates for major world currencies, top news, and key events impacting price movement.

🔎 Search Analytics Page (Scale Plan and above)



We've launched a new search analytics feature that provides insights into the performance of search engines over time. It allows tracking and analyzing key metrics such as average response speed, total number of searches, success rate, and error rate. The data is visualized through interactive charts, displaying daily search volumes and breaking down performance for average speed and success rate every hour. This feature is available for Scale plans and higher.

🔧 Engine Improvements:


Google Search API:

  • Added a new parsing field - Questions and Answers.
  • Added a new parsing field - Events.
  • Fixed an issue where inline_images were not parsed correctly.

Bing Search API:

  • Fixed a Bing API request issue that caused timeouts.

Google Maps API:

  • Removed the ll request parameter from the list of required parameters.

Amazon Product API:

  • Fixed an issue where the specifications_list was not parsed correctly.

Google Finance API:

  • Fixed an issue where the graph results displayed default results for 1 month (1M) instead of 1 day (1D).

Google Images API:

  • Now returns full image titles instead of shortened ones.

Other improvements:

  • Added enterprise plans.
  • Sped up the Search History page by adding missing database indexes.
  • Added "Time Taken" column in Search History.

⏭️ What's coming next?


  • Google Maps Reviews API
  • YouTube Reviews API
  • YouTube Channel API
  • YouTube Trending API
  • Playground
  • Organisational features

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

March 2024: Update

New
Today, we're excited to share the latest improvements and give you a sneak peek into what’s coming next.

🌟 Freshly Released Engines:





Google Maps allows searching for businesses using latitude and longitude GPS coordinates. Collect addresses, ratings, phone numbers, and websites. Improve local SEO by tracking business listing positions, and gathering ad information.

The Google Patents API enables retrieval of patent information, including titles, abstracts, claims, and descriptions. This API is crucial for research, product development, and competitive analysis.

The Google Patents Details API offers detailed information on individual patents, such as publication and application dates, inventors, and assignees.

The YouTube Video API allows for searching videos, retrieving video details, and accessing titles, descriptions, and related videos. 

🔄 Early Renewal




We've introduced a new feature that automatically renews your plan if all plan searches are used. This ensures uninterrupted access to our APIs and services so you can continue your work without any issues. No need to wait until the end of your billing cycle – we've got you covered.

🔗 LangFlow Integration:


​LangFlow​ is a dynamic graph where each node is an executable unit. Its modular and interactive design fosters rapid experimentation and prototyping, pushing hard on the limits of creativity. We have added a custom component allowing you to call any engine we support. We are looking forward to adding more examples and tutorials about it in the future. 

🔧 Engine Improvements:


Google Trends API:

  • Completely refactored the engine. JSON keys were left unchanged. You can now enjoy stable and fast response times without any errors.
  • Fixed `now 7-d` parameter support.

Google Search API:

  • Improved parsing speed on mobile devices. There was a bug causing significantly higher response times compared to desktop.
  • Fixed ​About this result field.

Google News API:

  • Fixed top stories extraction.

Bing API:

  • Fixed an issue causing some requests to return an error status.

YouTube API:

  • Fixed an issue causing several requests to return error status.

YouTube Transcripts API:


Other improvements:

  • Fixed documentation anchor links. Parameters can now be selected as anchor links as well.
  • Improved sidebar navigation in documentation pages.

⏭️ What's coming next?


  • Google Maps Places API
  • Google Maps Reviews API
  • Google Maps Photos API
  • Google Finance API

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

February 2024: Update

New
This month, we’ve rolled out several updates that we think you’ll find valuable: a new integration, significant engine and performance improvements, and a fresh search history feature.

🌟 ​Bing Search Engine​


We're excited to release Bing Search API offering organic results, ads, local results, related questions, and related searches. Similarly to Google, it supports precise location-based targeting using our ​Locations API. If that's not enough, you can also use latitude and longitude coordinates for coordinate-level precision. 

To ease integration, JSON keys align closely with those of the Google Search API. Perfect for building tools across different search engines.

🚀 Major Performance Upgrade


We've optimized our API, focusing on speed and stability. Key steps included revamping our proxy network on January 4th and boosting server capacity while fine-tuning our proxy network on a per-engine basis on January 12th, thanks to December’s backend efforts on the admin side.

As a result, SearchApi average response time is already less than 3 seconds. We anticipate improving it further to bring it below the 2-second threshold.

🔎 Search History Page


A new feature to review recent searches with the ability to filter by the search engine. We hope it will be really useful in debugging automated queries.

🔧 Engine Improvements:


Google Search API:

  • Added a new major parsing field: Sports Results. This includes three different types of sports results: 
    • Athlete Stats: This type can be triggered when searching for player statistics.
    • Team Sports: This type can be triggered when looking for games in basketball, football, or other team sports.
    • Single Sports: This type can be triggered when searching for golf, F1, tennis, or other individual sports games.



Google Shopping API:

  • Fixed product link and added `prds` value when there is no `product_id`.
  • Fixed documentation num parameter, as Google Shopping does not support the num=10 parameter and returns 60 results instead.

Google Product API:

  • Fixed an issue where the selected Product didn't return the main image of it.

Google Scholar API:

  • Fixed issues setting `num=20`.

Google Lens API:

  • Added mobile support as Google Lens returns larger images on it.
  • Added exact price and currency to visual_matches.
  • Added price, and stock information to exact_matches. Fixed source.
  • Fixed HTML. Now HTML shows an image, which was searched for as well.
  • Added ​country​ parameter. It allows making searches in a specific country to influence shopping results and the accuracy of place identification. 

Google News API:

  • Fixed top stories extraction.

Google Trends API:

  • Google Trends API is fully functional again.

Youtube Transcripts API:

  • API now returns all available languages even if the transcripts are not available for selected languages. Check ​documentation for an API example.

Amazon Product API:

  • Added bundle items & bundle carousel.
  • Improved variants extraction.
  • Improved buybox extraction.
  • Improved compare_with_similar table parsing.

🔗 New Integrations:


  • ​search_with_lepton​ - A dynamic, LLM-supported conversational search tool with an intuitive interface and shareable, cached results for efficient information retrieval.

Other improvements:


  • Transitioned to Stripe Embedded Checkout.
  • Upgraded to Ruby 3.3 in our production environment.

⏭️ What are we working on next?


  • Google Maps API
  • YouTube Channel API
  • Amazon Reviews API
  • Consider a CORS policy update for easier API experimentation.

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

January 2024: Update

New
Happy New Year! We're excited to start 2024 by improving your experience with SearchApi. This month, we've worked hard to improve our engines, especially Google. We believe these improvements will make your work easier and help you discover new things more efficiently.

🔧 Engine Improvements:


Google:

  • Added "Discussions and Forums". The element shows forums and their topic based on your query.
  • Implemented a new case of sitelinks - list - for organic results.
  • Added new element "From sources across the web" that shows sites that cover a specific topic in this grid list view.
  • Fixed local_ads and ads on mobile and desktop.
  • Fixed inline_images.
  • Fixed top_stories.
  • Added weather_results on mobile for up-to-date weather information.
  • Fixed the snippet in organic_results. Google started testing longer snippets, which might be related. 

Google News:

  • Fixed top_stories.

Google Shopping:

  • Fixed an issue when filters didn't return any text.
  • Added Google Shopping pagination.

Google Product:

  • Fixed an issue where typical_prices didn't return any results.

Google Lens:

  • New fields are being parsed - videos, related_searches, and exact_matches.

👨🏻‍💻 Product Changes


  • Enabled Link, Apple Pay & Cash App in the checkout. 
  • Added a status page to display uptime and performance metrics.
  • Released the first version of the ​Press Page. It provides marketing material for the affiliate program.

⏭️ What are we working on next?


  • Testing Bing API & Google Maps API that will be released this month.
  • Work in progress for API usage stats and search history features, initially for the admin side, with plans to extend to user accounts.
  • Even better API response time. We are in the process of optimizing our proxy pool. Thanks to the work done on the admin side. 
  • More integrations. 

Stay tuned for more updates! As always, we're committed to providing you with the best experience. Got feedback or questions? Just hit reply.

December 2023: Update

New
We're excited to bring you our latest updates for this month, including two new integrations, a major engine update, and an affiliate program.

🌟 New Engines


This is a single reason our update is late this month. We wanted to polish and release it before Christmas. 🎄 

Use the Amazon Product API to scrape product details, prices, delivery information, ratings, reviews, and more. It supports 22 Amazon domains and retrieves product information by ASIN.

We made sure to parse as much information as possible on the page and we think this is one the most comprehensive parsers on the market right now. We will still polish it during the week, so expect some minor changes. 

Customizing the delivery location country and zipcode is one of the key features we will be exploring at the start of 2024 and expanding the Amazon support further. 


Use the Google Autocomplete API to scrape real-time search autocomplete suggestions, their relevancy, popularity, and type. It supports nine different clients, including Chrome, Firefox, and Safari, each offering unique autocomplete results. Google Autocomplete API comes with a customizable cursor position, allowing different variations of suggestions.

💰 Affiliate Program (30% revenue share)


Promote SearchApi and earn a 30% revenue share for every new customer you refer.

We partnered with Rewardful as our affiliate software. We thoroughly tested our system, so it correctly tracks all people registering on the platform, even through the Google and Github logins. Learn more about the program on the ​affiliate program page​.

After signing up you get immediate access to your personal affiliate link and promotion materials, which you can use to recommend the SearchApi. Building integrations, self-hosted tools, and writing or organizing webinars are just a few ways to get started. Head over to the ​affiliate page​ for more ideas.

🔧 Engine Improvements:


Google

  • The new "Things to Do" block displays the most popular places to visit in the selected location.




Google Scholar

  • Improved snippets to return full text hidden under "show more" in DOI or full article title searches.




🔗 Integrations


  • ​​SerpBear​ - a simple, self-hostable, open-source SERP position tracking app. It allows you to track your website's keyword positions in Google and get notified of their positions.
  • ​Langchain​ (Python) - when integrating SearchApi into the Langflow, we noticed some issues, mainly with imports. This ​PR​ fixed the issues and has been released in ​the v0.0.346​ version.
  • ​HayStack​ - LLM orchestration framework to build customizable, production-ready LLM applications. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search, or conversational agent chatbots.

Other changes

  • Improved accessibility on the landing and documentation pages.
  • Improved loading speed for the documentation pages.
  • SEO improvements: markup & semantic tags.


⏭️ What are we working on next?

  • Development of Bing Search API and Google Maps API.
  • A status page to display our uptime and performance metrics.
  • Work in progress for API usage stats and search history features, initially for the admin side, with plans to extend to user accounts.
  • More integrations are underway. 

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

November 2023 Update

New
As another month wraps up, we're eager to share our latest strides in enhancing your development workflow.

🛠️ Developer Experience


  • Integration Examples. Expanded support for 13 programming languages, enhancing our library support to facilitate seamless integration with SearchApi. We've introduced intuitive icons for quick actions, enabling you to copy code snippets or open API examples in a new tab more efficiently.
​​



  • API Key Management
    • Multiple API Keys. ​The new Account API keys page​ allows the creation of distinct API keys for different environments or individual team members, reinforcing your operational security posture.
    • Zero downtime rotation. Transition between old and new API keys without service interruption, ensuring continuous operation.
    • Default API key. The default API key is used sitewide across all integration examples, so you can test the API faster.
    • The default API key is now hidden in the dashboard. 

🔧 Engine Improvements


Google
  • New "Salary Estimates" block providing valuable compensation data.



  • New "About this result" block offering contextual information about search results.



  • "People also search for" block is now parsed as `related_searches` on mobile.
  • Added pagination for Google searches, News, and Jobs.

Google Lens
  • Now supports query parameters in image URLs, correcting previously encountered encoding issues.

👨‍💻 DevOps


  • Resolved 504 HTTP response timeouts by adjusting the CloudFront to accommodate longer wait times for AWS ALB responses and established CloudWatch alerts for proactive monitoring.
  • Upgraded our infrastructure to utilize Rails 7.1.1 & Ruby 3.2.2 for improved performance and stability.

🔗 New Integrations


  • ​Flowise​ Integration. A new addition to facilitate low-code/no-code development for LLM applications.
  • ​Pipedream​ Integration. Enables connection to over 1000+ APIs with options for both coders and non-coders. Detailed information is available on our ​SearchApi Pipedream page​.

⏭️ What's coming next?


  • Amazon Product API
  • Google Maps API
  • TikTok API
  • More integrations

Your engagement is crucial for our continuous improvement. We encourage you to test these updates and look forward to your valuable feedback. Thank you for being an integral part of the SearchApi developer community.

October 2023 Update

New
We're thrilled to update you on some of the newest enhancements and give you a sneak peek into what’s coming next.

🦜🔗 ​LangChain​ Integration

Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. However, using these LLMs in isolation is often insufficient for creating a truly powerful app - the real power comes when you can combine them with other sources of computation or knowledge.

By embedding the SearchAPI, the Large Language Model can respond by querying real-time data to answer questions about current events. Its usage is particularly interesting with the "Self-ask with search" agent. This agent is equivalent to the original ​self-ask with search paper​, where a Google search API was provided as the tool.

Another common user case is summarizing long documents that naturally run into context window limitations. Here is an example ​using SearchApi to load a lengthy YouTube video transcript​, then summarize and distill it with a refine chain.

SearchApi is now natively available as part of ​langchainjs​ and ​langchain​ libraries.

For a quick overview, check out this tweet on SearchApi integration in LangChainJS.

Here is the official SearchApi documentation on its usage in Python and Javascript:
  •  Python examples​
  •  JavaScript/TypeScript examples​​

🌟 Freshly Released Engines:

  ✔️  Google Lens API​ - shipped in less than 24 hours based on ​@levelsio tweet on Twitter​. Useful as a tool in ​affiliate marketing​ for fetching product sources, links, prices, and stock information.
  ✔️  YouTube Transcripts API​
  ✔️  Amazon Search API​

🔧 Engine Improvements:

Google
  •  Added the ability to select a custom date range.
  •  Fixed `inline_videos` parsing on mobile
  •  Added `source` to desktop & mobile organic results
  •  Fixed `detected_location` in `search_information`
  •  Fixed `date` and `snippet` identification in organic results

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Product engine
  •  Integration examples in multiple languages and libraries
  •  LangChain integration (Ruby)

Keep an eye out for further updates! We're dedicated to ensuring you have the best experience. 

September 2023 update

New
During the last month, we added 6 new engines, made significant improvements to the API response time, and made sure it is highly scalable.

🌟 Freshly Released Engines

  ✔️  Google Product Offers​
  ✔️ ​ Google Product Reviews​
  ✔️  Google Product Specifications​
  ✔️  Google Scholar​
  ✔️  Google Scholar Cite​
  ✔️  Google Scholar Profiles​

🔧 Engine Improvements

Google
Introduced an `organic_results` type ​Answer Box field for desktop devices.

Google Play Store
Resolved an issue with the next_page_token. When searching for a specific application, the next_page_token will now reliably return the second page of Similar Products, eliminating previous errors.

Google Trends
Added the ability to select a custom date range.

🚀 Performance Improvements

  • Optimized all key database calls. (Thanks to NewRelic)
  • Refactored API usage tracking. This will allow us to add usage charts.
  • Identified a blocking operation in our API and made it async. Saved `~70ms` on API response time.
  • Performed load testing. We managed to solve all the application and infrastructure issues.
  • Experimented with Rust for HTML parsing. It allowed us to improve the parsing time by up to 10x. This is still very much WIP.

⏭️ What's coming next?

  •  Google Maps engine
  •  Amazon Search engine
  •  YouTube Transcripts engine 
  •  Integration examples in multiple languages and libraries 
  • Langchain integration (Python) 

Stay tuned for more updates! As always, we're committed to providing you with the best experience.

August 2023 Update

New
The past month has been another period of high productivity for our team, with considerable upgrades predominantly made to our Google engines. Keep an eye out for more refinements as we persist in adapting and broadening our services in response to your valuable feedback and requirements.

🆕 New Features Released

✔️ Released Google Jobs engine
✔️ Released Google Videos engine
✔️ Released Google Images engine
✔️ Released Google Shopping engine
✔️ Released Google Product engine

⚡ Performance & DevOps

• Improved parsing time up to 4x
• Improved performance for all queries utilizing location geo-targeting. We are able to achieve geo-targetting consistently in 10ms-20ms (took up to 1s previously).
• Integrated CDN with 450+ Points of Presence and 13 regional edge caches in 90+ cities across 49 countries
• Zero-Downtime deployment (Blue-green)
• Integrated Continous Delivery process into our team workflow
• Upgraded infrastructure components to the newest versions available.
• Integrated NewRelic & connected to AWS for enhanced performance monitoring

⏭️ What's coming next?

The main focus is on expanding our selection of available engines and increasing the performance even more. The goal is to have the average response time below 2 seconds consistently. We are happy that a significant part of Google searches are below 1 second already.

• Google Product Offers engine
• Google Product Reviews engine
• Google Product Specifications engine
• Google Maps engine
• Google Scholar engine
• Integration examples in multiple languages and libraries
• More performance improvements

We appreciate you choosing our services. We're working hard to make things better for you. If you need help or have questions, just let us know. We're always here. Thanks for your support.

July 2023 Update

New
Our team has had another highly productive month, with significant updates rolling out primarily on our Google Search API. One of the key enhancements is the improved support for mobile and tablet devices across numerous fields. This marks a major leap in our commitment to delivering a seamless and user-friendly search experience on all platforms, making it more convenient and efficient for users to find the information they need. Stay tuned for further improvements as we continue to refine and expand our services based on your feedback and needs.

🆕 New Features Released

✔️ Released Google News engine
✔️ Released Youtube engine

Significant Google Search Engine Update

We're excited to announce that we've significantly expanded our parsing capabilities within the Google Search Engine. The update will incorporate several new fields, mobile and tablet devices support.

New fields in Google Search Engine:

• Jobs
• Local Map
• Inline Tweets
• Inline Recipes
• Inline Images

Mobile and tablet is now supported for given fields:

• Inline Videos
• Jobs
• Top Stories
• Inline Tweets
• Ads
• Local Ads

Additional Updates

In order to further enhance your experience, we have:

• Fixed curl examples
• Changed product_info key in shopping_ads to extensions
• Changed documentation API key description
• In our continuous effort to enhance user experience, we have revised the titles of our API examples

⏭️ What's coming next?

• Google Jobs engine
• Google Maps engine
• Google Videos engine
• Google Images engine
• Google Scholar engine
• Google Shopping engine
• Google Search engine organic results tablet and mobile devices support


Your choice to use our services means a lot to us. We're dedicated to making your experience better. If you need help or have questions, please reach out. We're always here to help. Thanks again for your support.



June 2023 Update

New
We have completed another productive month with an emphasis on improving and broadening our Google Search Engine capabilities. Our team has added parsers for more than 100 distinct page blocks, reflecting our ongoing commitment to delivering unparalleled data quality and precision.

🆕 New Features Released

✔️ Released Google Trends engine
✔️ Released Google Trends Autocomplete engine
✔️ Released Google Locations API engine, which enables Google searches with high-precision coordinates across 195 supported countries.

Significant Google Engine Update

We've broadened our parsing scope in the Google engine to include the following fields:

• Ads;
• Local Ads;
• Shopping Ads;
• Inline Videos;
• Inline Shopping;
• Inline Recipes;
• Related Questions;
• Local Results;
• Top Stories;
• Knowledge Graph (with emphasis on Business, Shopping, and Jobs entities).

Additional Updates

In order to further enhance your experience, we have:

• Integrated Intercom live chat support to offer instant assistance.
• Automated embedding of the api_key in all documentation examples when you are signed in for ease of use.
• Resolved issues with product_highlights extraction in Google Play Store.

We thank you for choosing our services and we're committed to continuously improving and providing the best experience for you. If you have any questions or need further assistance, please do not hesitate to reach out.

May 2023 Update

New
🚀 Official Launch Announcement

Though we're a new face in the SaaS world, we're far from inexperienced. For years, we've been providing advanced web scraping solutions with high-performance APIs for our private clients. Now, we're thrilled to make our services available to everyone.

Our mission is simple: to make public data accessible to everyone, with precision, quality, and rich, extensive information at the core of everything we do, while ensuring our APIs perform efficiently and reliably.

Join us on this journey, and as a new startup, we pledge to deliver unparalleled technical support and direct communication with our engineers. This commitment, coupled with our focus on API performance, sets us apart and remains our number one priority and unique value proposition.

🆕 New Features Released

✔️ Google Play Store engine.
✔️ Google Play Product engine.
✔️ Initial version of Google search engine. We parse organic_results and related_queries on the desktop.
✔️ 100 free searches - newly registered users can test our platform with 100 complimentary searches.

⏭️ What's coming next?

• Significant Google search engine improvements. This includes precise search by geolocation and a bunch of new blocks rolling over the next weeks.
• Trends Search API
• Youtube Search API
• Infrastructure upgrades - we're enhancing our infrastructure and conducting stress tests to accommodate even larger scale projects.

Join us today and experience the difference of our advanced web scraping solutions. Welcome aboard!