1. 2018-02-06 - Configure Git Credentials; Tags: Configure Git Credentials
    Loading...

    Configure Git Credentials

    Gitlab and Github offers personal access tokens for git access over https. They are the only accepted method of authentication when you have Two-Factor Authentication (2FA) enabled. Since I have a Yubikey, I have to use a personal access token, if SSH is not viable, e.g. working in safe guarded environment. A token however has the advantage that it can expire, thus forcing me to exchange it more frequently to hinder attack scenarios.

    It is quite uncomfortable to enter user and token for every git operation on the remote repository. Git offers a credential storage, to simplify that. This article demonstrates my setup for Gitlab. You can use it for any other git hosting provider, like AWS CodeCommit or Bitbucket.

    First step is to configure a credential store as file. While it can reside in the .gitconfig itself, putting in a separate file seems the better approach. Configure git to use a file. This file could also locate in a encrypted filesystem, which home usually is.

    git config --global credential.helper 'store --file ~/.my-credentials'

    The authentication is cached for 900 seconds (15 minutes), see also https://git-scm.com/docs/git-credential-cache/2.8.0.

    Now I have to fill the credential storage with my gitlab access token. Using git credentials command to store the authentication data. Important is to use the file storage by omitting the respective option. git will read the input from stdin and stores it to the given file. A blank line ends the input from stdin.

    class="language-bash">git credential-store --file ~/.my-credentials store protocol=https host=gitlab.com username=cinhtau password=a-magic-password

    Now I can operate (pull/push) on my git repositories without the hassle to enter user and password :+1:.

    Another example for AWS CodeCommit. Change to your respective region and credentials.

    git credential-store --file ~/.my-credentials store
    protocol=https
    host=git-codecommit.us-east-1.amazonaws.com
    username=cinhtau-at-4711
    password=another-magic-password 

    Example for Github, generate Token under Developer settingsPersonal access tokens

    git credential-store --file ~/.my-credentials store
    protocol=https
    host=github.com
    username=cinhtau
    password=mapper-magic
    git

    Comments


    Leave a comment


  2. 2017-07-24 - Update Jekyll blog entry meta-data; Tags: Update Jekyll blog entry meta-data
    Loading...

    Update Jekyll blog entry meta-data

    If you track your jekyll post with git, preferably if you hosting your github pages, than you can utilize the git attributes to automatically update post meta-data. For instance we replace the modified information in the yaml frontmatter (that is the jekyll metadata) every time you do a update on a jekyll post or page.

    General Mechanism

    Check Customizing Git - Git Attributes and Customizing Git - Git Hooks to understand the mechanism in detail.

    In short:

    • write a script that replaces the date with current date
    • configure git to call that filter on every commit
    • setup filter for respective jekyll files, i.e. markdown

    Replacing Modified Timestamp

    Before we I configure the git hook, I have to find the appropriate command to do that. Therefore create an example yaml frontmatter.

    >tan@omega:~$ cat test.txt --- layout: post status: publish published: true title: Reindex Watcher Indices with Curator author: tan date: '14.07.2017 10:48' modified: '2017-07-14 16:58:43 +0200' categories: - IT tags: - elasticsearch ---

    Use sed to replace existing field with the new timestamp.

    sed -i "s/modified:.*/modified: \'$(date +'%Y-%m-%d %H:%M:%S %z')\'/" test.txt
    

    Check the modified output

    tan@omega:~$ cat test.txt
    ---
    layout: post
    status: publish
    published: true
    title: Reindex Watcher Indices with Curator
    author: tan
    date: '14.07.2017 10:48'
    modified: '2017-07-25 10:36:08 +0200'
    categories:
    - IT
    tags:
    - elasticsearch
    ---
    

    The git manual has a ruby program example

    tan@omega:~/bin$ cat expand_date 
    #! /usr/bin/env ruby
    data = STDIN.read
    last_date = `git log --pretty=format:"%ad" -1`
    puts data.gsub('$Date$', '$Date: ' + last_date.to_s + '$')
    

    Filters

    It turns out that you can write your own filters for doing substitutions in files on commit/checkout. These are called “clean” and “smudge” filters.

    • smudge - before they’re checked out (on checkout)
    • clean - before they’re staged in (on add)

    The manual uses the smudge filter for the date expansion, but in my case it must be clean. I want the date substituted with the current timestamp during the commit.

    This must be done in the directory where the git repository is.

    Filters from Example:

    $ git config filter.dater.smudge expand_date
    $ git config filter.dater.clean 'perl -pe "s/\\\$Date[^\\\$]*\\\$/\\\$Date\\\$/"'
    

    Check with gitconfig -l if the settings are applied.

    Setup Git Attributes

    Create in the respective git repository the .gitattributes file. To apply the filter to all jekyll files, add the file extension for markdown and assign the filter, which was previously configured.

    tan@omega:~/Sources/cinhtau.github.io$ cat .gitattributes 
    *.md filter=dater
    

    The basic problem is that filters only work on checkout and add. Following the example I always had to remove and make a checkout the file to get the date substitution. This is a very extensive effort.

    To modify a file on commit mostly the pre-commit hook is the right way to accomplish this.

    Pre-Commit Hook

    Git has examples files in the .git/hooks directory. Based on that I wrote this simple pre-commit script. It must be named pre-commit and must be placed in the hooks directory.

    #!/bin/bash
    
    if git rev-parse --verify HEAD >/dev/null 2>&1 ; then
       against=HEAD
    else
       # Initial commit: diff against an empty tree object
       against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
    fi
    
    staged_files=`git diff-index --name-status --cached $against      | # Find all staged files
                    egrep -i '^(A|M).*\.(md)$' 		                  | # Only process jekyll files
                    sed -e 's/^[AM][[:space:]]*//'                    | # Remove leading git info
                    sort                                              | # Remove duplicates
                    uniq`
    
    
    partially_staged_files=`git status --porcelain --untracked-files=no | # Find all staged files
                            egrep -i '^(A|M)M '                         | # Filter only partially staged files
                            sed -e 's/^[AM]M[[:space:]]*//'             | # Remove leading git info
                            sort                                        | # Remove duplicates
                            uniq`
    
    # Merge staged files and partially staged files
    staged_and_partially_staged_files=${staged_files}$'\n'${partially_staged_files}
    
    # Remove all files that are staged *AND* partially staged
    # Thus we get only the fully staged files
    fully_staged_files=`echo "$staged_and_partially_staged_files" | sort | uniq -u`
    
    for FILE in $fully_staged_files ; do
        # substitute every jekyll modified timestamp with current timestamp
        sed -i "1,10s/modified:.*/modified: \'$(date +'%Y-%m-%d %H:%M:%S %z')\'/" "$FILE"
    done
    

    Pay attention, to replace only the modified in the frontmatter, sed is limited to lines 1 to 10. Ensure that modified is listed in this range.

    sed -i "1,10s/modified:.*/modified: \'$(date +'%Y-%m-%d %H:%M:%S %z')\'/" "$FILE"
    

    Testing

    Now let’s do some testing. Show modified files.

    tan@omega:~/sources/cinhtau.github.io$ git status
    On branch master
    Your branch is up-to-date with 'origin/master'.
    Changes to be committed:
      (use "git reset HEAD <file>..." to unstage)
    
            renamed:    scripts/prepare-commit-jekyll -> scripts/pre-commit-jekyll
    
    Changes not staged for commit:
      (use "git add <file>..." to update what will be committed)
      (use "git checkout -- <file>..." to discard changes in working directory)
    
            modified:   .gitattributes
            modified:   Gemfile        
            modified:   _posts/2017/07/2017-07-17-update-jekyll-meta-data.md
    

    Commit the jekyll post

    tan@omega:~/sources/cinhtau.github.io$ git commit _posts/2017/07/2017-07-17-update-jekyll-meta-data.md -m "test pre-commit"
    [master cefec94] test pre-commit
     1 file changed, 11 insertions(+), 7 deletions(-)
    

    Check the date substitution. :raised_hands:

    tan@omega:~/sources/cinhtau.github.io$ head _posts/2017/07/2017-07-17-update-jekyll-meta-data.md
    ---
    layout: post
    published: true
    title: Update Jekyll blog entry meta-data
    author: cinhtau
    date: '24.07.2017 07:49'
    modified: '2017-07-25 10:36:08 +0200'
    categories:
    - IT
    tags:
    
  3. 2016-04-28 - Configure cntlm as git proxy; Tags: Configure cntlm as git proxy
    Loading...

    Configure cntlm as git proxy

    Every three months, security compliance requires to change the password. This also applies for the proxy authentication. If you have a bunch of tools, like Eclipse, IntelliJ or any other applications that needs proxy auth for updates and access, it can be a hassle to change the settings in all applications. For Windows, cntlm is a relief. It stands between your applications and the corporate proxy, adding NTLM authentication on-the-fly. So basically you change the auth data only for cntlm, and all other applications using it to bypass the proxy.

    The git configuration for one example project before:

    vinh@omega MINGW64 /c/dev/src/neframa (master)
    $ git config -l
    core.symlinks=false
    core.autocrlf=input
    core.fscache=true
    color.diff=auto
    color.status=auto
    color.branch=auto
    color.interactive=true
    help.format=html
    http.sslcainfo=C:/Program Files/Git/mingw64/ssl/certs/ca-bundle.crt
    diff.astextplain.textconv=astextplain
    rebase.autosquash=true
    user.email=cinhtau@gmail.com
    user.name=cinhtau
    gui.encoding=utf-8
    core.autocrlf=true
    https.proxy=https://vinh:secret23@proxy.cinhtau.net:8080
    http.proxy=http://vinh:secret23@proxy.cinhtau.net:8080
    core.repositoryformatversion=0
    core.filemode=false
    core.bare=false
    core.logallrefupdates=true
    core.symlinks=false
    core.ignorecase=true
    core.hidedotfiles=dotGitOnly
    ..
    

    As you can see there is still the proxy user, password and proxy server with respective port in the http(s).proxy setting. First we need to cleanup. Unset the settings

    git config --global --unset http.proxy
    git config --global --unset https.proxy
    

    The authentication data goes to the cntlm.ini configuration:

    #
    # Cntlm Authentication Proxy Configuration
    #
    # NOTE: all values are parsed literally, do NOT escape spaces,
    # do not quote. Use 0600 perms if you use plaintext password.
    #
    Username	vinh
    Domain		cinhtau.net
    PassNTLMv2      6F8811AD2E996CGG8495B4870E23AD74
    Proxy		proxy.cinhtau.net:8080
    NoProxy		localhost, 127.0.0.*, 10.*, 192.168.*, *.cinhtau.net
    Listen		3128
    Auth		NTLMv2
    

    Instead of

    git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080
    

    We use only this

    git config --global http.proxy http://localhost:3128
    git config --global https.proxy https://localhost:3128
    

    Now if you work with cntlm, you will have some output like this

    ******* Round 2 C: 6, S: 7 (authok=0, noauth=0) *******
    Reading headers (7)...
    HEAD: HTTP/1.1 200 Connection established
    Sending headers (6)...
    Ok CONNECT response. Tunneling...
    tunnel: select cli: 6, srv: 7
    forward_request: palive=0, authok=1, ntlm=0, closed=0
    Thread finished.
    proxy_thread: request rc = 0xffffffff
    forward_request: palive=0, authok=1, ntlm=0, closed=0
    Thread finished.
    proxy_thread: request rc = 0xffffffff
    Joining thread 537209432; rc: 0
    Joining thread 537141160; rc: 0
    
    git
  4. 2016-03-03 - Getting started with JBoss EAP Quickstarts; Tags: Getting started with JBoss EAP Quickstarts
    Loading...

    Getting started with JBoss EAP Quickstarts

    This post demonstrates how to setup the quickstart examples for the stable release JBoss EAP 6.4. The quickstart projects offer a variety of examples for the usage of Java EE 6 with JBoss.

    First of all, checkout the project from GitHub. Open the shell or gitbash on windows:

    $ git clone https://github.com/jboss-developer/jboss-eap-quickstarts.git
    Cloning into 'jboss-eap-quickstarts'...
    remote: Counting objects: 101739, done.
    remote: Compressing objects: 100% (33/33), done.
    remote: Total 101739 (delta 19), reused 4 (delta 4), pack-reused 101702
    Receiving objects: 100% (101739/101739), 48.58 MiB | 635.00 KiB/s, done.
    Resolving deltas: 100% (41761/41761), done.
    Checking connectivity... done.
    Checking out files: 100% (3198/3198), done.
    

    List tags

    vinnie@W30060 MINGW64 /c/dev/src/jboss-eap-quickstarts (7.0.x-develop)
    $ git tag
    1.0.0.Alpha2
    1.0.0.Alpha3
    1.0.0.M1
    1.0.0.M2
    1.0.0.M2b
    1.0.0.M2c
    1.0.0.M3
    1.0.0.M3b
    1.0.0.M4
    1.0.0.M5
    1.0.0.M6
    1.0.0.M7
    1.0.0.M8
    2.0.0.ER8
    6.2.0.Beta1
    6.2.0.GA
    6.3.0.GA
    6.4.0.GA
    6.4.0.develop
    7.0.0.CR1
    7.0.0.Final
    7.0.0.Final-subsystem
    7.0.1.Final-subsystem
    7.0.2.CR1
    7.0.2.CR1-javaee
    7.0.2.CR2
    7.0.2.CR3
    7.0.2.CR4
    7.1.0.Beta1
    7.1.0.Final
    7.1.1.CR1
    7.1.1.CR2
    7.1.1.Final
    7.1.2.M1
    EAP-6.2.0.ER4
    EAP_7.0.0.Beta1
    jdf-2.0.0.CR3
    jdf-2.0.0.Final
    jdf-2.0.0.M3
    jdf-2.1.0.Final
    jdf-2.1.1.Final
    jdf-2.1.2.Final
    jdf-2.1.5.Final
    jdf-2.1.6.Final
    jdf-2.1.7.Final
    jdf-2.1.8.Final
    jdf-2.1.9.Final
    jdf-eap6.1-initial-merge
    jdf-eap6.1-update2
    master-6.2.0.ER4
    master-6.2.0.ER5
    

    Update to dedicated release, tag: 6.4.0.GA for example

    vinnie@W30060 MINGW64 /c/dev/src/jboss-eap-quickstarts (7.0.x-develop)
    $ git checkout tags/6.4.0.GA
    Checking out files: 100% (3206/3206), done.
    Note: checking out 'tags/6.4.0.GA'.
    You are in 'detached HEAD' state. You can look around, make experimental
    changes and commit them, and you can discard any commits you make in this
    state without impacting any branches by performing another checkout.
    If you want to create a new branch to retain commits you create, you may
    do so (now or later) by using -b with the checkout command again. Example:
      git checkout -b <new-branch-name>
    HEAD is now at 71a4719... Update versions for 6.4.0.GA release
    

    If you are working in a company network, you might add the proxy to the delivered settings.xml.

    <proxies>
    <proxy>
                <id>yourProxy</id>
                <active>true</active>
    <protocol>http</protocol>
                <username>vinh</username>
    <password>secret</password>
                <host>proxy.cinhtau.net</host>
    <port>8080</port>
                <nonProxyHosts>localhost| 127.0.0.*| 10.*| 192.168.*</nonProxyHosts>
            </proxy>
        </proxies>
    

    Read more at https://maven.apache.org/guides/mini/guide-proxies.html, if you have further questions about proxies with Apache Maven. Build the project with Apache Maven 3 and the settings.xml from RedHat and skip the tests. You may deactivate all active or default activated profiles, if you don’t met the requirement.

    mvn clean install -s settings.xml -Dmaven.test.skip=true
    

    After that you run or deploy the generated Java EE artifacts to the JBoss EAP 6.4.x versions. Have fun. If you want to start with a quick project setup, e.g. a simple web application (war)

    mvn archetype:generate \
     -DarchetypeGroupId=org.jboss.archetype.eap \
     -DarchetypeArtifactId=jboss-javaee6-webapp-ear-blank-archetype \
     -DarchetypeVersion=6.4.0.GA -s settings.xml
    
  5. 2015-09-30 - Update to a specific tag with git; Tags: Update to a specific tag with git
    Loading...

    Update to a specific tag with git

    Git has the ability to tag specific points in history as being important. Typically people use this functionality to mark release points (v1.0, and so on). Following example demonstrated how to switch to a specific tag with git.

    Show available tags for Dropwizard Metrics (output shortened)

    $ git tag
    v1.0.0
    v3.0.0-BETA1
    v3.0.0-BETA2
    v3.0.0-BETA3
    v3.0.0-RC1
    v3.0.1
    v3.0.2
    v3.0.2-dropwizard
    v3.1.0
    v3.1.1
    v3.1.2
    

    Update to tag v3.1.2 without creating a branch

    $ git checkout v3.1.2
    Note: checking out 'v3.1.2'.
    You are in 'detached HEAD' state. You can look around, make experimental
    changes and commit them, and you can discard any commits you make in this
    state without impacting any branches by performing another checkout.
    If you want to create a new branch to retain commits you create, you may
    do so (now or later) by using -b with the checkout command again. Example:
      git checkout -b new_branch_name
    HEAD is now at c4b64c5... [maven-release-plugin] prepare release v3.1.2
    
    git
  6. 2015-07-07 - Configure proxy server for git; Tags: Configure proxy server for git
    Loading...

    Configure proxy server for git

    Pay attention that your credentials are stored in the config. A better way is to use ssh keys.

    Setup proxy server for HTTP and HTTPS

    git config --global http.proxy http://user:passwd@proxy.cinhtau.net:8080
    git config --global https.proxy https://user:passwd@proxy.cinhtau.net:8080
    

    Remove proxy server for HTTP and HTTPS

    git config --global --unset http.proxy
    git config --global --unset https.proxy
    
  7. 2015-07-03 - Using git under Windows with SSH; Tags: Using git under Windows with SSH
    Loading...

    Using git under Windows with SSH

    Git provides with the git bash and gui a sufficient way to manage git repositories. Using git with SSH causes an error. Somehow git doesn’t take Putty’s plink. Well the culprit was a former installed program - tortoisegit. This article about Git on Windows fixed my problem with tortoisegit. Just set the GIT_SSH environment variable to plink.

  8. 2015-07-03 - Switching remote URLs from HTTPS to SSH; Tags: Switching remote URLs from HTTPS to SSH
    Loading...

    Switching remote URLs from HTTPS to SSH

    If you access the git repository via HTTPS you have to provide each time authentication data. Switching to SSH is more convenient and secure. After adding your public SSH key to the repository in GitHub (or any other) you can access the repository with SSH. To switch your local git repository:

    # show current remote urls
    git remote -v
    # switch remote
    git remote set-url origin git@github.com/user/repository.git
    # check again
    git remote -v
    

    Replace user and repository and you are done. You may consider that other providers like Atlassian Stash or Bitbucket have another syntax.

    # Bitbucket example with user
    git remote set-url origin ssh://git@bitbucket.org:user/repository.git
    # Atlassian Stash example with default ssh port
    git remote set-url origin ssh://git@stashhost:7999/repository.git
    
  9. 2015-07-03 - Unknown host accessing Atlassian Stash; Tags: Unknown host accessing Atlassian Stash
    Loading...

    Unknown host accessing Atlassian Stash

    Atlassian Stash default server setting for SSH is port 7999. If you switch the git repository to SSH access, your client may reject the SSH connection.

    To resolve this try to login with ssh to the host

    Linux

    ssh gitstash:7999
    

    On Windows with plink (Putty Link)

    $ "C:Program Files (x86)PuTTYplink.exe" gitstash -P 7999
    The server's host key is not cached in the registry. You
    have no guarantee that the server is the computer you
    think it is.
    The server's rsa2 key fingerprint is:
    ssh-rsa 2048 ..
    If you trust this host, enter "y" to add the key to
    PuTTY's cache and carry on connecting.
    If you want to carry on connecting just once, without
    adding the key to the cache, enter "n".
    If you do not trust this host, press Return to abandon the
    connection.
    Store key in cache? (y/n) y
    
  10. 2015-07-03 - Push existing code to remote git repository; Tags: Push existing code to remote git repository
    Loading...

    Push existing code to remote git repository

    For the situation you have created a remote (master) repository on GitHub, BitBucket or OpenHub.

    cd existing-project
    git init
    git add --all
    git commit -m "Initial Commit"
    git remote add origin https://github.com/user/project.git
    git push origin master
    

    Steps explained:

    • Change to source directory
    • Initialize git repository
    • Add and commit the changes (ignore unwanted files with .gitignore)
    • Add remote repository, replace user and project with your data
    • Push to repository with branch master, you may have to authenticate yourself at the server or ssh-key
    git