Showing posts with label gitlab. Show all posts
Showing posts with label gitlab. Show all posts

Friday, July 6, 2018

Grouping of users in Gitlab

Leave a Comment

I have a set of users who are common to all of the groups and projects in the company. Presently, I have to add those "common members" individually to each and every group or project. Is there any way to group them by a name, thereby adding them to groups or projects?

For example, A, B anc C are the common members and can be grouped by name X, so that X can be added, instead of individually adding A, B and C

PS: Users are from LDAP

1 Answers

Answers 1

Users are from LDAP

Aynthing LDAP related would be, interm of group synchronization, related to GitLab EE (Enterprise Edition)

But, if you can simply define main group for your common team, and subgroup for your specific teams/projects, you can use EE/CE Namespaces

In GitLab, a namespace is a unique name to be used as a user name, a group name, or a subgroup name.

For example, consider a user named Alex:

(Here you can replace 'alex-team' by 'common-team', or any other more descriptive name)

By doing so:

  • Any team member mentions Alex with @alex
  • Alex mentions everyone from their team with @alex-team
  • Alex mentions only the marketing team with @alex-team/marketing
Read More

Invoking Webhook from GitLab returns SSL error

Leave a Comment

When creating a Webhook in GitLab 11.0.2 and testing it, I get this back this error:

Hook execution failed: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: tlsv1 alert internal error 

The Webhook URI is using HTTPS with a public certificate (not self signed).

SSL verification is disabled for this Webhook.

Update

I upgraded openssl1.0.2g to openssl1.0.2o but the error remains.

Then I tried to run:

openssl s_client -connect mywebhookhost:443 

That resulted in:

depth=1 C = BE, O = GlobalSign nv-sa, CN = GlobalSign Organization Validation CA - SHA256 - G2 verify error:num=20:unable to get local issuer certificate 

Update 2

/opt/gitlab/embedded/bin/ruby -ropenssl -e 'puts OpenSSL::OPENSSL_VERSION'

returns

OpenSSL 1.0.2o 27 Mar 2018

Update 3

GlobalSign CA certs are installed

 awk -v cmd='openssl x509 -noout -subject' ' >     /BEGIN/{close(cmd)};{print | cmd}' < /etc/ssl/certs/ca-certificates.crt | grep GlobalSign 

results in

subject=OU = GlobalSign ECC Root CA - R4, O = GlobalSign, CN = GlobalSign subject=OU = GlobalSign ECC Root CA - R5, O = GlobalSign, CN = GlobalSign subject=C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA subject=OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign subject=OU = GlobalSign Root CA - R3, O = GlobalSign, CN = GlobalSign 

2 Answers

Answers 1

From this thread, it looks like your openssl is too old

Have been fighting with TLS for a few days, realised my GitLab was running on an old debian8, upgraded to debian9. So now.

python -c "import ssl; print ssl.OPENSSL_VERSION" OpenSSL 1.1.0f 25 May 2017

So start checking/upgrading openssl, for your webhook script to run properly.

Answers 2

I have the same problem and to solve that i need to install root certificates on gitlab server.

See here how to.

Read More

Thursday, May 17, 2018

error: cannot overwrite multiple values with a single value

Leave a Comment

I want to change my git mergetool kdiff3 to p4merge. because I'm getting an error on my windows system using kdiff3 mergetool.

/mingw32/libexec/git-core/git-mergetool--lib: line 128: C:\Program Files\KDiff3\kdiff3: cannot execute binary file: Exec format error application/config/constants.php seems unchanged.

So that I want to change to kdiff3 to p4merge, Here also I'm getting an error like

warning: merge.tool has multiple values error: cannot overwrite multiple values with a single value Use a regexp, --add or --replace-all to change merge.tool.

enter image description here

How can I solve this problem? Either kdiff3 or p4merge

1 Answers

Answers 1

It is possible your kdiff3 installation is broken since it is not working. Or maybe you tried to edit config file manually and messed its content. why? Because windows executables have .exe extension in general. you may try editing config again.

Anyways, that is not important anymore. This is what you need to use if you want to try any other tool.

git mergetool --tool=p4merge 

There are possibly others already installed with your git. You can see all of them in addition to compatible others.

git mergetool --tool-help 

Edit: This command works only if you set your path to the tool correctly. Otherwise, you always get No files need merging result. You already know how to set the path, but I will include here for anyone else that might need.

Get the list of configuration items:

git config -l 

See if you already have values set correctly, if any. then set to correct value or remove.

git config --unset mergetool.p4merge.path git config --add mergetool.p4merge.path "c:/somewhere/p4merge.exe" 
Read More

Saturday, April 21, 2018

How to deploy node app to remote host from Jenkins?

Leave a Comment

This is Jenkins file in the root of node app directory:

pipeline {     agent any     triggers {         pollSCM('* * * * *')     }     stages {         stage("deploy") {             steps {             sh "scp"             }         }     }  } 

I configured Jenkins to connect to remote gitlab node proj repo to checkout node project along with Jenkinsfile and run the project's Jenkinsfile. This part works fine but what to do now to perform (note that Jenkins server and the server on which node js is running as well as gitlab repo are all remote to each other):

run these commands on remote server on which node app is running  cd ~/mynodeproj   pm2 stop mynodeproj   copy project source files from Jenkins server to remote server where  node app is running   npm install   export NODE_ENV=production   pm2 start mynodeproj 

How to achieve this?

Do I need to setup private/public keypair on server running jenkins so that jenkins server can do scp to copy file to remote server running node app?

3 Answers

Answers 1

I'm suggest we can use Rocketeer for this case.

Here is Rocketeer tree on my Jenkins Server for NodeJS App

$ tree .rocketeer/ .rocketeer/ ├── config.php ├── hooks.php ├── logs │   ├── develop--20170613.log │   ├── develop--20170614.log │   ├── develop--20170616.log │   ├── staging--20180323.log │   ├── staging--20180324.log │   ├── staging--20180326.log │   ├── production--20180223.log │   ├── production--20180226.log │   ├── production--20180227.log │   ├── production--20180227.log │   └── custom-environment--20180328.log ├── paths.php ├── remote.php ├── scm.php ├── stages.php └── strategies.php 
  • You can manage remote environment of NodeJS App: Develop, Staging, Production (at config.php file)
  • It's will pull newest source code on your Gitlab and keep releases version like Capistrano (at remote.php file)
  • It's can run your pm2 command line after deployed newest source code (at hooks.php file)
  • It's already help run npm install NodeJS packages.

So here is my Jenkins Job setting:

Source Code Management

enter image description here

Build Triggers

enter image description here

Build

enter image description here

#!/bin/bash -el cd $JENKINS_HOME/app-deploy/app-socketio rocketeer deploy --on="develop" 

develop mean connect to Develop Remote Server (at .rocketeer\config.php file)

'connections'      => [     'develop' => [         'host'      => '35.xx.xx.xx',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/foo.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ],     'staging' => [         'host'      => '34.xx.xx.xx',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/bar.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,      ],     'production' => [         'host'      => '18.xx.xx.xx:63612',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/woot.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ],     'custom-environment' => [         'host'      => '13.xx.xx.xx:63612',         'username'  => 'ec2-user',         'password'  => '',         'key'       => '/var/lib/jenkins/.ssh/test.pem',         'keyphrase' => '',         'agent'     => '',         'db_role'   => true,     ], ], 

And run pm2 command line configure at hooks.php file

'after'  => [     'setup'   => [],     'deploy'  => [             "pm2 delete mynodeproj", //Delete old pm2 task             "pm2 start src/mynodeproj.js", //Start new mynodeproj     ],     'cleanup' => [], ], 

Hope this can help you!!

Answers 2

What to do now to perform (note that Jenkins server and the server on which node js is running as well as gitlab repo are all remote to each other):

Those command should be in a file part of your Git repo, meaning checked out in the workspace used by the pipeline.

From there, you can scp that script to the server:

sh 'scp -r script user@server:/path/server' 

And execute it through ssh, provided the private/public key of the remote user is deployed on the agent (or passed through the ssh-agent plugin):

sh 'ssh user@server /path/server/script' 

The trick though, as mentioned here, is that the PATH of an ssh session is fairly limited, and might not include node, if node is not installed in a standard path like /usr/bin.
So you might want to modify your script in order to use absolute paths for your executable (like /absolute/path/to/node)

Answers 3

Your question is how to deploy your application to remote host from jenkins

easiest solution

1.using Jenkins File

If your using a pipeline project, then you will notice a Pipeline Syntax option, just select ssh Publisher:send artifacts over SSH (you'llneed the plugin Publish over SSH plugin)

enter image description here Add an SSH server

  • Source Files :mention the files you wish to transfer (**/*.war takes the WAR file only)
  • Remove prefix : target(WAR files will generally be located in a target folder so i'm telling I don't want the target folder to be copied only my WAR)
  • Remote Directory : Leave it blank( since your taking it from your project)
  • Exec command : Commands to be executed after copying the files( like starting tomcat or unzip any properties files and place at a particular location - in your case the commands to start node from your server)

Finally just click Generate Pipeline Script and BOOM, you get the script (it internally does an scp) just copy and paste it into your Pipeline file and your done :)

enter image description here)

2.Publish over SSH

You can do the same thing( follow the above steps adding details only) in a normal Jenkins project in the Post Build Action If you don't have a Pipeline Project (helps if you don't have/know a Pipeline Project)

enter image description here

Do I need to setup private/public keypair on server running jenkins so that jenkins server can do scp to copy file to remote server running node app?

Yes its required(Recommended) you can add the deatils as shown below in Jenkins under Manage Jenkins, Configure Jenkins Under Publish over SSH and you can test your connection .

enter image description here

Hope this was useful :)

Read More

Wednesday, October 11, 2017

Configuring GitLab to use SAML OmniAuth with an Active Directory IdP

Leave a Comment

I am in the process of altering an existing GitLab installation to use SAML rather than LDAP for authentication.

At this point, users can successfully sign into the Web application using the 'Sign in with Saml' button. I am unclear, however, about what seems to be a difference between the LDAP and SAML approaches: users with accounts created via an LDAP sign-in can then access Git repositories (e.g. using clone, push, ...) using their LDAP usernames and passwords, but users with accounts created via a SAML sign-in cannot.

Through experimentation, I’ve found that users can access the Git repositories if they use the GitLab UI to set a separate GitLab account password on the account that is created during the initial SAML interaction. I was pointed in this direction by a GitLab message that appeared after creating a project under one of the new user accounts: 'You won't be able to pull or push project code via HTTPS until you set a password on your account'.

It seems possible that this separate password configuration is only necessary because I’ve misconfigured the SAML integration somehow. So, my question is whether I am wrong to expect that authenticating access to the GitLab-hosted Git repositories would work the same regardless of whether SAML or LDAP is used? If not, does anyone know of relevant SAML configuration settings that I should review?

In case it’s of interest: I have posted a similar question to the GitLab Google group, but I have not received any responses there yet.

0 Answers

Read More

Friday, May 5, 2017

Docker: How to use selenium server to do nightwatchJS test?

Leave a Comment

I don't know how to run a selenium server with my NodeJS application, whose files are located in the ./bundle folder of the custom e2e:latest docker image.

I think I have to add the selenium server and webdriver chrome into the Dockerfile for the e2e:latest image, don't I?

This is what I have done so far:

I've created a java:8-jre based docker image with NodeJS and nightwatchJS:

Dockerfile

FROM java:8-jre  ## Node.js setup RUN curl -sL https://deb.nodesource.com/setup_4.x | bash - RUN apt-get install -y nodejs  ## Nightwatch RUN npm install -g nightwatch 

This image is then used for the test:

gitlab-ci.yml

build:   stage: build   tags:     - deploy   script:     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY     - meteor npm install --production     - meteor build $PACKAGE_PATH --directory     # Maybe something like...? - docker build -t $CI_REGISTRY_IMAGE:e2e .  nightwatch:   image: e2e:latest   stage: e2e   tags:     - e2e   before_script:     - cd ./bundle   script:     - nightwatch 

The configuration looks like this:

nightwatch.conf.js

module.exports = {     'src_folders'           : ['test/e2e'],     'output_folder'         : 'reports',     'custom_commands_path'  : '',     'custom_assertions_path': '',     'page_objects_path'     : '',     'globals_path'          : '',     'test_runner'           : {         'type'   : 'mocha',         'options': {             'ui'      : 'bdd',             'reporter': 'list'         }     },      'selenium': {         'start_process': false,         'server_path'  : '',         'log_path'     : '',         'host'         : '127.0.0.1',         'port'         : 4444,         'cli_args'     : {             'webdriver.chrome.driver': './bin/chromedriver'         }     },      'test_settings': {         'default': {             'launch_url'   : 'http://localhost',             'selenium_port': 4444,             'selenium_host': 'localhost',             'silent'       : true,             'screenshots'  : {                 'enabled': true,                 'path'   : 'reports/error-screenshots'             },             'desiredCapabilities': {                 'browserName'      : 'chrome',                 'javascriptEnabled': true,                 'acceptSslCerts'   : true             }         },          'chrome': {             'desiredCapabilities': {                 'browserName'      : 'chrome',                 'javascriptEnabled': true,                 'acceptSslCerts'   : true             }         }     } } 

1 Answers

Answers 1

Not sure whether this is suitable for Gitlab CI but take a look at Selenoid project. This is a small (6 Mb) binary that launches browsers in separate Docker containers or by starting Webdriver process directly. So if container approach is not suitable for your needs try to pack Selenoid + e.g. Chromedriver + Chrome to the same container with Node.js. No need to install Java when using Selenoid.

Read More

Monday, April 18, 2016

GitLab Activity section shows recent changes with a big delay

Leave a Comment

Since I have upgraded our GitLab CE to 8.6 It stopped showing the recent changes for example in Activity section also not showing latest commits for MR's.

Right now running the up to date version (8.6.4) still the same issue.

As you can see below the Check looks fine

root@server:~# sudo gitlab-rake gitlab:check Checking GitLab Shell ...  GitLab Shell version >= 2.6.12 ? ... OK (2.6.12) Repo base directory exists? ... yes Repo base directory is a symlink? ... no Repo base owned by git:git? ... yes Repo base access is drwxrws---? ... yes hooks directories in repos are links: ...  Administrator / projectX ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Check directories and files:      /var/opt/gitlab/git-data/repositories: OK     /var/opt/gitlab/.ssh/authorized_keys: OK Test redis-cli executable: redis-cli 2.8.24 Send ping to redis server: PONG gitlab-shell self-check successful  Checking GitLab Shell ... Finished  Checking Sidekiq ...  Running? ... yes Number of Sidekiq processes ... 1  Checking Sidekiq ... Finished  Checking Reply by email ...  Reply by email is disabled in config/gitlab.yml  Checking Reply by email ... Finished  Checking LDAP ...  LDAP is disabled in config/gitlab.yml  Checking LDAP ... Finished  Checking GitLab ...  Git configured with autocrlf=input? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config outdated? ... no Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory setup correctly? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) projects have namespace: ...  Administrator / projectX ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.1.0 ? ... yes (2.1.8) Your git bin path is "/opt/gitlab/embedded/bin/git" Git version >= 2.7.3 ? ... yes (2.7.4) Active users: 21  Checking GitLab ... Finished 

Tried to:

  1. Restart GitLab
  2. Clear cache

No help...

0 Answers

Read More

Tuesday, April 12, 2016

Jenkins: Link between Stacktrace and GitLab

Leave a Comment

We use Jenkins and GitLab in our company. We use py.test to create the XML output for Jenkins which gets rendered by the jUnit Jenkins Plugin.

The stacktraces of exceptions are pure ascii up to now. It would be really great if we could hook into Jenkins somehow and show hyperlinks to our gitlab server instead of a html pre block.

In our case we it would be enough to filter the every line of the test output and use a regular expression on it....

Example output of the Jenkins jUnit Plugin:

File "/home/modwork_ems_d66/src/foo/foo/utils/testutils.py", line 975, in wrapped     return fn(*args, **kwargs) File "/home/modwork_ems_d66/src/foo/foo/tests/FooTest.py", line 641, in test_empty_models_if_unused     models)) 

I want a hyperlink to the server hosting the git repo of "foo". In our case this would be "https://source/repos/foo/files/master/foo/foo/tests/FooTest.py"

How to get this done? I know how to use regular expressions. The question is how to hook the regex into jenkins.

Here is a screenshot to visualize it enter image description here

I want to this link on the TestReport of a single test. The URL pattern of these pages:

https://jenkins/job/ci_foo_bar/lastCompletedBuild/testReport/src.myapp.myapp.tests.test_tree/TestCase/test_something/ 

1 Answers

Answers 1

If you can acquire exception output after testing process you can implement a parser which prepare a simple html report for you as a build-step.

A a simplest way you can implement just a regular expression with sed like

sed -e 's@"/home/modwork_ems_d66/src\(.*\?\)"@<a href="https://source/repos/foo/files/master\1">\1</a>@' exceptions.txt 

which operates on the exception file.

Read More