Create a for-each in .gitlab-ci to run multiple variables

Working with GitLab is awesome this tool has a lot of features and I really like it, one of the advantages is its big community and documentation also there are tons of ideas to improve this tool.

Recently working in a pipeline I realized that it could be great to have a variable like a list in order to have something like a while or a for-each sadly I couldn’t see something similar to use.

For that reason, I made this simple workaround, pass a variable like a string then create a document like a txt and finally apply a xagrs command.

Challenge.

Create a standard CI/CD for python projects those projects use different packages from many sources for that reason it is necessary to repeat many times the curl instruction.

Sample of .gitlab-ci with multiple lines to reapet the curl

• before_script:
• #download and package content
  • – curl sS -o paquete1.tar http://www.regner.com.mx/instalando-jenkins-en-una-vm-centos/
  • – curl sS -o paquete2.tar https://stackoverflow.com/questions/8445445/
  • – curl sS -o paquete3.tar https://stackoverflow.com/questions/

Solution.

Create a variable to set those different url’s and having a simple “while”

1. create our variable.
2. create a txt file with a valid format to be sorted
3. apply the curl instruction in each line

#variable in one single line \n  is used to have new rows 
NECESARYPACKAGES: -sS -o paquete1.tar http://www.regner.com.mx/instalando-jenkins-en-una-vm-centos/ \n -sS -o paquete2.tar https://stackoverflow.com/questions/8445445/ \n -sS -o https://stackoverflow.com/questions/

Having a variable we could set multiple rows or instructions separated by \n

Now we are going to pass the content of that variable as a txt file. with a simple echo

 - echo -e "${NECESARYPACKAGES}" >> packsgen.txt

the result of this is a packsgen.txt with this content:

sS -o paquete1.tar http://www.regner.com.mx/instalando-jenkins-en-una-vm-centos
sS -o paquete2.tar https://stackoverflow.com/questions/8445445
sS -o paquete3.tar https://stackoverflow.com/questions/

Time to apply our curl sentence.

- xargs -n4 curl < packsgen.txt

-n4 is used because our lines are composed each 4 components sS -o tarfile URL if you don’t have this way to separate each line you could get an error because xargs + curl will take the file content as a single row.

Try by your self:

In a console or terminal window.

echo -e "-sS -o paquete1.tar http://www.regner.com.mx/instalando-jenkins-en-una-vm-centos/ \n-sS -o paquete2.tar https://stackoverflow.com/questions/8445445/ \n-sS -o paquete3.tar https://stackoverflow.com/questions/" >> packsgen.txt
xargs -n4 curl < packsgen.txt

This has to produce these files: 3 packages.tar and 1 txt file.

In the end, you could parametrize the content in one variable to reuse code and your Gitlab-ci will appear nice.