> ## Documentation Index > Fetch the complete documentation index at: https://docs.clarky.ai/llms.txt > Use this file to discover all available pages before exploring further. # Websites > Crawl your site so every page becomes a document your agent can use ## Connecting a website When you create an agent from a domain, Clarky crawls the site right away. If your agent has no website yet, **Context > Websites** shows a box to connect one: enter the domain and Clarky starts the first crawl. We follow links on your domain (up to 250 pages), keep the readable text of each page and drop navigation, images, scripts and forms. ## Pages are documents Every crawled page shows up in **Context > Websites** (and in Documents under the Website filter) as its own document with the page title and URL. You can: * **Search** pages by title or URL * **Edit** the captured text (re-crawling replaces it) * **Toggle channels** per page, or **Apply channels to all pages** from the site card * **Disable** individual pages, or use the **Use website pages** switch to turn the whole site off and on * **Re-scrape page** from a row's menu to refresh just that page * **Remove** a page (it returns on the next crawl if it still exists) Website pages default to Chat, Text and Email. Voice is off for pages: voice prompts are fixed at call start, so a whole site cannot be included. Put the essentials in a text document instead. ## Re-crawling Click **Re-crawl** on the site card whenever your website changes. Pages that still exist are refreshed in place and keep the channels and settings you chose; pages that disappeared from your site are removed when the crawl finishes. Your existing knowledge stays live while the crawl runs. Re-crawls are limited to once every 6 hours and are available on paid plans. ## Create a document from the website **Documents > Add > Create from website** summarizes all crawled pages into a new pinned *About \* document. Useful right after a crawl or a big site update. See how pages fit alongside text, files and Q\&A