Stop Google Crawling Your Cart and Filter Parameter URLs with AI

Keep faceted-filter and cart URLs out of Google so your real product pages get the crawl budget.

Difficulty Beginner

Time ~10 min

Tools run_wp_cli

Prerequisites

A connected WooCommerce site with an SEO plugin that has a robots.txt editor (Rank Math, Yoast, or AIOSEO).

The prompt

Review my site's robots.txt through my SEO plugin and stop search engines from crawling my WooCommerce cart, checkout, account, and filter and sort parameter URLs. [My store also uses these custom filter parameters: list any, for example ?pa_color= or ?stock_status=.]

Keep WooCommerce's existing default rules, then add Disallow rules for /cart/, /checkout/, /my-account/, and the sort and faceted-filter parameters (orderby, filter_, min_price, max_price, add-to-cart, rating_filter). Show me the complete robots.txt you plan to save and wait for my OK before writing it. Do not change anything until I confirm.

After saving, fetch my live /robots.txt back and paste it so I can confirm the rules are live. Finally, remind me which URLs I should submit to the Search Console Removals tool, since robots.txt only stops future crawling and will not clear pages Google has already indexed.

Already connected to WPVibe?

Skip the copy-paste. Just ask your AI for the “Stop Google Crawling Your Cart and Filter Parameter URLs with AI” recipe, or describe the task, and WPVibe runs these steps on your own site.

A WooCommerce store quietly spawns thousands of URLs you never made on purpose. Every sort order, price slider, color filter, and add-to-cart link becomes its own crawlable address, and Google dutifully indexes the lot. They pile up in your Search Console coverage report as duplicate or “indexed, though blocked” pages, they waste the crawl budget that should go to real products, and most store owners only notice once the warnings are already stacked up.

Ask your AI assistant and WPVibe reads the robots.txt your SEO plugin already serves, then adds Disallow rules for the cart, checkout, and account pages plus the faceted-filter and sort parameters, all while keeping WooCommerce’s own protective defaults. It shows you the full file before anything is saved and writes it through Rank Math, Yoast, or AIOSEO, so there is no raw file editing and nothing to break. The change is reversible, and your previous rules are one prompt away.

Paste the prompt, name your store platform and any custom filter parameters you use, and ask for the rule list back so you can spot-check before it saves. One caution worth knowing: blocking a URL in robots.txt stops future crawling but does not remove pages Google already indexed, so the prompt reminds you to clear those with the Search Console Removals tool. Give it a day, then recheck your coverage report.

Outcome

Your robots.txt blocks cart, checkout, account, and filter and sort parameter URLs while keeping WooCommerce defaults, so search engines stop crawling near-duplicate store URLs.