Log in

Sign up

Webmaster's panel

Categories.

Didn't find a solution to your problem? Contact us on live online chat or ask on Facebook group.

Missing any feature? Share your ideas with us.

Homepage / SEO and marketing / Robots.txt file edition

WebWave drag and drop website builder lets you create unique websites. You can build your website from scratch by starting with a blank page, or choose various website templates from our free collection and modify them according to your needs. With WebWave website builder, you have a complete website design and hosting system at your fingertips.

This website was created with WebWave.

Find us online

Website builder.

Service level availability

Offer.

Company.

Data Processing Agreement

Help.

Useful articles.

How to make a website

Small Business Website

Simple Website Templates

Modern Website Templates

Best HTML website builder

Best responsive website builder

No-Code Website Builder

Canva vs WebWave

Wix vs WebWave for SEO

What is a white label

How to become a web designer

Affiliate Marketing

AI web design

ChatGPT web design

Midjourney web design

Blog

18 June 2021

Robots.txt file edition

Configuration of your mailbox in Gmail

Configuration of your mailbox in MacOS & iOS

Configuration of your mailbox in Outlook

Configuration of your mailbox in Thunderbird

Create a mailbox

E-mail in WebWave

How to change the mailbox capacity

Import and export e-mails

Reset mailbox password

Server details needed to configure your email client

Sign in to your mailbox

The Robots.txt file reports the information to indexing robots on which pages and files on your website they have to index. It is used mainly in order the website not to be overloaded by requests and helps manage the website's traffic. It is used to exclude webpage or file from Google indexing.

How to edit Ropots.txt file

Exemplary commands

How to edit Robots.txt file

1. Go to the SEO Settings

2. Then go the "Robots.txt" tab.

3. Enter your code to the field "Content of the robots.txt file"

Exemplary commands

User-agent: *

The code that specifies the indexing robots and the content to be indexed. * Mark means that all the indexing robots will be blocked, but the very use of googlebot would mean that only the Google indexing robots will be blocked.

Disallow: /gallery

Exclusion of the gallery subpage from indexing.

Disallow: /lib/jieq5l/IMG_1898-k2z3myc6.jpeg

Exclusion from indexing of a file uploaded to media library. The exact file path one may find under the miniature in "My images and files" in "Settings".