README.md

@@ -1,61 +1,42 @@
 # Help Center Spider
+## About
+This is a spider tool with which you can visit all links on https://docs.otc.t-systems.com to find urls that are not correct.
 
-The Open Telekom Cloud Helpcenter Spider is a spider tool visiting all
+## Requirements
-links starting from its landing page on https://docs.otc.t-systems.com/
+After you cloned the repository you need to prepare an environment to run the tool. You can easily do this with 
-to find and identify urls that are not correct. It parses all types of
+python virtual environment:
-hyperlinks and normalizes them into a canonical format. The spider
-descents into the document tree via [...] bredth or width first search.
-[and does what?] [when is logged which event?] 
-
-## Getting started
-
-Once you installed the code and its required packages into an virtual
-environment and checked its configuration file `config.json`, the web
-spider starts invoking the tool without any arguments. Results are
-listed in [... TBD].
-
-## Requirements and Installation
-After you cloned this repository you need to prepare an environment to
-run the tool. You can easily do this with a Python virtual environment:
 
 ```
-$ cd _local_folder_/
+$ cd <local_folder>/
-$ git clone https://gitea.eco.tsi-dev.otc-service.com/infra/hc-spider.git
-$ cd hc-spider
 $ python -m venv venv/
-$ source venv/bin/activate
+$ python -m pip install -r requirements.txt
-(venv)$ python -m pip install -r requirements.txt
 ```
 
 ## Configuration
-In _config.json_ you can define several items:
+In _config.json_ you can define a couple items:
 
-- _watchdog_file_: if you run the tool in the background and want to
+- _watchdog_file_: if you run the tool in the background and want to stop it properly (not using `kill`), 
-  stop it properly (without sending a signal with `kill`),  just send
+just send an exit message into the watchdog file: `echo exit > watchdog.fifo`
-  an exit message into the watchdog file: `echo exit > watchdog.fifo`.
+- _timer_runtime_: maximum runtime limit in seconds
-- _timer_runtime_: maximum runtime limit in seconds.
+- _log_dir_: logging folder
-- _log_dir_: logging folder.
+- _logging_interval_: frequency of dumping log files
-- _logging_interval_: frequency of dumping log files.
+- _workers_: number of workers (background processes) you want to run. If you set to 0 it will count from the number of cores (_number_of_cores_ - 1)
-- _workers_: number of workers (background processes) you want to run.
-  If you set to 0 it will count from the number of cores
-  (_number_of_cores_ - 1)
 - _starting_point_: base url where to start
 
-## Operations
+## How to run
-There are two ways to start the spider:
+There are two ways to do it
 
-### In the foreground
+### In foreground
 ```
 $ source venv/bin/activate
-(venv)$ python main.py
+$ python main.py
 ```
 
-### In the background
+### In background
 ```
 $ source venv/bin/activate
-(venv)$ nohup python main.py > log/hc_spider.log 2> log/hc_spider.err <&- &
+$ nohup python main.py > log/hc_spider.log 2> log/hc_spider.err <&- &
 ```
 
-### Stopping the process polietely
+In case you running the tool in background you can stop the execution with `$ echo exit > <watchdog_file>`
-To stop the tool when run in the background, send a command to the 
+
-control fifo with: `(venv)$ echo exit > _watchdog_file_`