0.2.0
LatestAdded
- Search engines and link-preview bots:
Googlebot,bingbot,Applebot,DuckDuckBot,facebookexternalhit,LinkedInBot,Twitterbot,WhatsAppandSlackbot-LinkExpanding(24 bots in total). Every bot belongs to a group (ai,search,social), and--botsaccepts group names. - Share-preview check:
og:title,og:description,og:image(downloaded: status, content type, size and dimensions for PNG, JPEG, GIF and WebP) andtwitter:card, with errors, warnings and tips. Link-preview bots that get a page without theog:imagea browser sees are flagged.--no-previewskips it. ai-bot-access verify <ip>: checks a request claiming to be a crawler against the operator's published IP list (OpenAI, Anthropic, Perplexity, Google, Microsoft, Apple, DuckDuckGo, Common Crawl, Amazon), with forward-confirmed reverse DNS as a fallback for Google, Microsoft, Apple and Amazon. Verdicts:verified,spoofed,unverifiable,unknown-bot.verifyBot()andcreateVerifier()in a newai-bot-access/verifyentry point with no Node-only imports, for Next.js middleware, edge functions and Express. Lists are cached for 24 hours.ai-bot-access logs <file...>: reads nginx/Apache combined logs and JSON lines (Cloudflare, Vercel and most custom formats), plain or gzipped, or stdin. Reports hits, successful and refused responses, distinct IPs and real/fake/unverifiable hits per bot, the most requested pages, and visitors sent by AI assistants. Flags crawlers refused on 20% or more of their genuine requests, and impostors.- How to fix: every report ends with concrete fixes grouped by problem, and the place to look on the host or CDN it recognises from the response headers (Cloudflare, Vercel, Netlify, Hostinger, Akamai, Fastly, CloudFront, Sucuri, WP Engine, Kinsta, SiteGround, Shopify, Wix, Squarespace).
- A
403for a bot whose operator publishes its IPs is reported as to confirm (unconfirmed: true): it is often fake-bot protection, which refuses a crawler's name from other addresses, exactly what the test sends. It no longer fails the run unless--strictis set.summary.unconfirmedcounts them. --list --jsonprints the bot table as JSON, including each bot's group and verification sources.- Automated releases: merging a
chore(release)pull request publishes to npm with provenance and creates the GitHub release from these notes.
Changed
- New report layout: bots grouped by kind with one clear status per bot, a result line, then Problems, To confirm, Blocked in robots.txt, Notes and How to fix. When problems are found, the last line links to a page with more help.
- A bot that does not always follow robots.txt and is disallowed there is reported with a note that it may fetch the page anyway.
- The page is read up to 1 MB (was 64 KB) so the share-preview tags are found on large pages.
- CI runs on
actions/checkoutandactions/setup-nodev7. - Contributing guide documents the commit and changelog rules. Pull request titles and changelog entries are checked in CI.
- Links to codevibe.in in the README, contributing guide and issue template carry UTM tags. The README on GitHub says
utm_source=github; the README published to npm saysutm_source=npm(switched by the newprepack/postpackscripts), so visits from each can be told apart. - The README opens with a 30-second preview (animated GIF) of the launch video, linking to the full 95-second video with sound on codevibe.in.