https://www.codrut.pro/tags/humanity/ Recent content in Humanity on Codrut Constantin Gusoi Hugo en Sat, 12 Aug 2023 05:50:06 +0000 https://www.codrut.pro/snippets/disallow-gpt-in-robots-txt/ Sat, 12 Aug 2023 05:50:06 +0000 https://www.codrut.pro/snippets/disallow-gpt-in-robots-txt/ <p>Enable <code>robots.txt</code> in your <code>hugo.toml</code> (formerly <code>config.toml</code>):</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-toml" data-lang="toml"><span style="display:flex;"><span><span style="color:#a6e22e">enableRobotsTXT</span> = <span style="color:#66d9ef">true</span> </span></span></code></pre></div><p>Add a file <code>layouts/robots.txt</code> with:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-txt" data-lang="txt"><span style="display:flex;"><span>User-agent: GPTBot </span></span><span style="display:flex;"><span>Disallow: / </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span>User-agent: * </span></span><span style="display:flex;"><span>Allow: / </span></span></code></pre></div>