Perplexity SPACE open source widens how agents fetch live web content
Perplexity released developer tools and open-sourced its SPACE sandbox, making it easier for contained AI agents to access pages and prompting publisher pushback.
Perplexity released developer tools and open-sourced its SPACE containment sandbox, widening how AI agents can fetch live web content. Publishers have responded by starting to restrict access to prevent unwanted scraping and automated reads.
What happened
Perplexity released new developer tools and made its SPACE sandbox open source, providing a packaged environment that makes it easier to run contained AI agents which can fetch live web content. The move lowers the technical barrier for teams and developers to deploy agents that browse pages and pull information in real time.
Open-sourcing SPACE means organisations can inspect, modify and run the sandboxed agent environment themselves rather than relying on a closed platform. That increases the number of ways agents can behave when they access external sites, including automated page reads, navigation and content extraction.
As agent access has become easier, some publishers have started to restrict access to parts of their sites to limit automated fetching. Those restrictions include technical controls and legal limits designed to stop agents scraping or using content without permission.
The change has broader industry context: Perplexity is already a large consumer and enterprise player, and tools that make agent-based access simpler are being watched closely by publishers and site operators concerned about control and data use.
Why it matters for businesses
When agents can fetch live pages, AI assistants and third-party services may read, summarise or quote site content directly rather than relying only on cached indexes or licensed data. That changes how a business’s pages might be discovered, cited or presented by assistants.
Open-source sandboxes make it easier for many operators to run agents under different configurations, including ones that bypass rate limits or mimic human browsing patterns. That raises a controllability problem for publishers who want to limit automated access to protect revenue or exclusive content.
Publishers restricting agent access means some sites may disappear from the outputs of AI assistants that rely on live fetching. For businesses this can reduce visibility in assistant answers unless they adapt technical and legal controls or provide clear, accessible data feeds for bots.
Finally, organisations that run sensitive systems should note how containerised or self-hosted agent environments change data flow. The same sandboxing that makes agents portable also makes it possible to keep sensitive workloads inside controlled infrastructure, but only if organisations adopt appropriate deployment and governance practices.
What the AI assistants say today
We checked multiple AI assistants to see whether they reflect the news that Perplexity open-sourced SPACE and that publishers are restricting agent access. Assistants partly reflect the broader theme that agents can fetch live web content and that site operators should use controls, but none fully captured all aspects of the story in our check.
Some assistants cited documentation and community pages about agent behaviour and controls. Our table summarises which assistants reflected the news and the kinds of sources they used.
| Assistant | Reflects the news? | What it said | Sites it cited |
|---|---|---|---|
| ChatGPT | Partly | The answer discusses risks and controls for AI agents fetching live content, which aligns with the news about Perplexity releasing tools and sandboxing agents, but it does not mention Perplexity, the SPACE open-source release, or publishers restricting access as reported. | none |
| Gemini | Partly | The answer notes Perplexity-related bots/agents and advises robots.txt, WAF, and ToS updates, which aligns with the news about Perplexity releasing developer tools and SPACE enabling agent access, but it does not mention Perplexity open-sourcing SPACE or publishers restricting access specifically, nor the Valarian/UK government partnership in the facts. | docs.perplexity.ai, perplexity.ai, ppc.land |
| Perplexity | Partly | The answer incorporates the news theme that agents can fetch live web content and recommends controls, but it does not mention Perplexity open-sourcing SPACE or the specific publisher reactions (restricting access) described in the news. | docs.perplexity.ai, geotoolbox.ai, github.com, maximuslabs.ai, news.ycombinator.com, offpistestudio.com |
| Google AI Overviews | Partly | The answer reflects the news about Perplexity releasing tools that enable live content fetching and prompting publishers to take technical and legal measures, but it does not mention the open-sourcing of the SPACE sandbox or the Valarian/UK government partnership and related facts from the news. | ainews.com, gagadget.com, hyscaler.com, itsfoss.com, nationalbusinessregister.co.uk, neowin.net |
What to do now
If your site or service could be read by agents that fetch live content, take these practical steps to reduce unwanted access and preserve how content is used by assistants.
- Audit public endpoints and pages to decide what can be freely fetched and what must remain protected.
- Implement technical controls such as robots.txt, rate limiting, IP blacklists, and a web application firewall to deter automated agents.
- Clarify legal terms of use and acceptable automated access in your site terms, and apply explicit licensing or API keys for permitted programmatic access.
- Offer a clean, documented API or data feed for partners and trusted assistants so they do not need to scrape HTML pages.
- Monitor traffic and use behavioural detection to identify agent-like access patterns, adapting blocks or allowances as needed.
- Consider an external review of how AI assistants cite or summarise your site; Vasukai AI Search Desk can help check how assistants talk about a business.
Questions
Does open-sourcing SPACE mean any agent can read my site?
Open-sourcing SPACE lowers the technical barrier for running contained agents that can fetch pages, but it does not by itself force any specific behaviour. Site operators can still use technical and legal controls to limit or block automated fetching.
Will AI assistants now quote my content without permission?
Assistants that fetch live content can read and summarise pages, which may result in quoted or cited material appearing in outputs. Some publishers have already begun restricting access to prevent scraping or unauthorised reuse.
Should I change robots.txt or other site settings because of these tools?
Yes. Updating robots.txt, enforcing rate limits, and using a WAF or IP controls are practical measures mentioned by AI assistant guidance and in responses to the rise of agent-based fetching. Legal terms and explicit APIs are also recommended.
Can organisations keep sensitive AI workloads isolated?
Yes. The broader industry trend includes options to run AI inside controlled infrastructure so sensitive workloads do not leave an organisation’s environment. That approach is one reason some vendors and partners promote self-hosted or sovereign deployments.
Sources
- Perplexity has an open-source alternative now, and I'm hooked, makeuseof.com
- AI giant Perplexity taps London startup Valarian in sovereign tech drive, cityam.com
Facts in this report come from the sources above; figures were checked against them. Drafted with AI and checked by the Kael Tripton editorial team. How we report.