DATAPREPREP · data as of 2026-09-14 · 3 min read한국어로 읽기
We evaluated LinkedIn's robots.txt against 13 URL types. 10 are allowed for Googlebot; Feed post (/feed/update/), Group, and Search results are disallowed. OpenAI's and Anthropic's search crawlers are also allowed on the Feed post (/feed/update/) URL, while PerplexityBot, ChatGPT-User, GPTBot, and Google-Extended are blocked from the whole site.
Two claims about LinkedIn circulate at the same time: that LinkedIn content ranks well on Google, and that LinkedIn posts don't get indexed. Both can be true, because LinkedIn writes different robots.txt rules for different URL patterns.
We fetched LinkedIn's robots.txt on 2026-09-14 and tested 13 public URL patterns against Google's and Bing's search crawlers and 6 AI crawlers. We did not scrape Google results pages; their robots.txt disallows crawlers.
Crawlers with identical results share a column. The other four AI crawlers are PerplexityBot, ChatGPT-User, GPTBot, and Google-Extended. Fetched on 2026-09-14.
Allowed: Member profile, Company page, Post (/posts/), Article (Pulse), Newsletter, Job posting, Event, School page, Showcase page, and Hashtag feed. Beyond member profiles and company pages, Googlebot may crawl articles, newsletters, job postings and event pages.
Each disallowed type is matched by one line: Feed post (/feed/update/) by "Disallow: /feed/update/"; Group by "Disallow: /groups/"; Search results by "Disallow: /search*".
LinkedIn robots.txt, lines from the OAI-SearchBot group · fetched 2026-09-14
The OAI-SearchBot group has "Disallow: /embed/feed/update/" but no "Disallow: /feed/update/" line, and the Claude-SearchBot group is the same. robots.txt is matched line by line, so that one line decides allowed or disallowed.
The same post can be shared under different URL patterns, and the pattern decides whether Googlebot may crawl it. The /posts/ pattern is allowed for Google, Bing, OpenAI and Anthropic search crawlers alike.
# LinkedIn may, in its discretion, permit certain automated access to certain LinkedIn pages,
# for the limited purpose of including content in approved publicly available search engines.
Header comment in LinkedIn's robots.txt · fetched 2026-09-14
The file names 76 user-agent groups and ends with a * group set to Disallow: /. Only named crawlers can crawl the paths assigned to them.
PerplexityBot, ChatGPT-User, GPTBot, and Google-Extended are each blocked with Disallow: / in their own group. Google's documentation says Google-Extended does not affect inclusion or ranking in Google Search, so that line is separate from how LinkedIn appears in Search.
The robots.txt has 0 Sitemap lines. We compare other platforms in a companion article on platform sitemaps.
So how do you get LinkedIn content in front of search and AI answers?
Put what you want found on company pages, member posts, articles or newsletters, not in Group.
When you share a post off LinkedIn, check that the link uses the /posts/ pattern. Google's and Bing's crawlers are disallowed from the Feed post (/feed/update/) pattern.
Keep originals that Perplexity or ChatGPT's user-triggered fetcher should read on your own site, and link to them from LinkedIn.
Methodology
Evaluated with RFC 9309: the group naming the user agent applies if present, otherwise the * group; the longest matching rule wins.
URLs are pattern examples with placeholder IDs and names. We did not fetch any LinkedIn page.
Fetched on 2026-09-14 with the user agent MarketResearchBot/0.1, robots.txt only. We did not check whether LinkedIn serves different files to different user agents.
Allowed in robots.txt means a crawler may fetch the URL. Login walls and noindex, which can still keep a page out of the index, are not part of this test.
LinkedIn's help center disallows our user agent in robots.txt, so we did not read it.