Category: development

Helper bash scripts to move projects from your small SSD to a larger HDD (and back)

I use a Dell 9020m Micro-PC as “terminal-server” for development. It has a small SSD, but also a larger HDD attached.

I’ve created two simple bash-scripts that help move projects I currently don’t work on from SSD to HDD. They add symlink to the “off-boarded” location so that I can basically keep on working on them, but with slower I/O. When I want to work on the project again (and need the I/O speed), the project is moved back to the SSD (“on-boarding”).

To make re-off-boarding fast, the on-boarded state is kept on the HDD and re-used when off-boarding the project again.

These two simple scripts have been released under MIT license on GitHub.

Add SSH host key fingerprint to Jenkins for Git checkouts

I have a self-hosted Gitea instance, and also operate my own Jenkins instance. On the Jenkins instance, strict host-key checking is enabled. When adding the first reference to a Git repository hosted on my server, the following error appears:

Failed to connect to repository : Command "git ls-remote -h -- ssh://git@<myserver>:22222/martin/jenkins-test-docker-pipeline.git HEAD" returned status code 128:
stdout:
stderr: No ECDSA host key is known for [myserver]:22222 and you have requested strict checking.
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

The reason is that since it’s the first time I’m accessing a repository on this server, so the SSH host fingerprint is not in the known_hosts file for this SSH connection. Since I run this installation of Jenkins inside a Docker container and I don’t want to manually edit files in the file-system, I rely on setting the appropriate settings in Manage Jenkins > Security > Git Host Key Verification Configuration. This is set to Manually Provided Keys.

The easy solution is to set it to Accept First Connection. But I want to be stay on the manual mode. The easiest way to get the SSH host fingerprint via ssh-keyscan (-p 2222 is for specifying the SSH server port, which is a non-standard port in my case):

ssh-keyscan -p 22222 myserver

The output looks like this:

# myserver:22222 SSH-2.0-OpenSSH_9.1
[myserver]:22222 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC085ixMnTlpr0pxXmkeJ6X479mbW/9PGDeUvD8hnG7EVUn3WsnnSG8yZkmU+jzg2W+xmFd7WIdaYLt6UcGvCS3RZIye68+qu64UToKX6CdTQOWyj6z9kd8tLoPBobsBd7tRyGaXU4c4UkCR5M44KhYtbQz0bgL7u+sL0z+R3lbOVyXaYPiSmUf/Wsd8fA2VcdWHkXJx0MMNMSVj/hgkZR7RfHzP4SZSqRLhn/AzIdx4DDuyGyPbVxu1ppnFtumRwlBkgat9UpMWkelREhcUdJtrZO1KPpA6DOkxIH8X/WtXyWToS9EjPb8FVTvzdjG2C4Zi0DkogH3no9vQcXLiihz
# myserver:22222 SSH-2.0-OpenSSH_9.1
[myserver]:22222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfTT9eEpDmd7ToGAorTW1X9uuJVhZl+KX9phmTpTy2e8U7l31jWn2TnKlXOp5oKgivpQ2cVjcTyazyrFB7MhgI=
# myserver:22222 SSH-2.0-OpenSSH_9.1
[myserver]:22222 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFoEzPpEWApszceLM/jWHvAbrTppjsTzftw79yTSS5Po
# myserver:22222 SSH-2.0-OpenSSH_9.1
# myserver:22222 SSH-2.0-OpenSSH_9.1

It only makes sense to copy the non-comment lines (the ones not starting with a # to the configuration).

Now Git checkouts to this repository should work, once you have configured the appropriate credentials.

cygwin & Play Framework / Typesafe Activator

I am currently playing around with using Play Framework on cygwin. I noticed that Typesafe Activator, which has replaced the play command in recent Play versions, ruins the mintty terminal of cygwin: you won’t see any echo of your keystrokes after activator returns.

A simple solution is to blindly type stty sane into the terminal, which will reset it. Another way is, to wrap the activator in a special cygwin-activator bash script:

#!/bin/bash
activator $@
R=$?
stty sane
exit $R

The power of git aliases

Based on a recent question on stackoverflow I found the power of git aliases and want so share one I invented for answering the question and after that I found very useful in everyday git use:

git config --global alias.add-commit '!git add -A && git commit'

After this, you can simply check in all new, modified, and deleted files with a simple

git add-commit -m 'My commit message'

I have aliased this command also to git ac in order to save further on typing. I never thought that this combination could be that useful, but actually I think it really is. Thanks to the questioner for bringing the idea up.

How to force Git to consider a file as binary

If you are using Git on Windows and follow my advise on how to get past the problem with the “suspicious patch lines”, you might run into problems if you are using Encapsulated PostScript (.eps) files in your repository.

PostScript files are almost plain-text files, and if you set core.autocrlf and core.safecrlf, they might cause problems with the EPS binary encoded parts, as they might be detected as text-files and therefore remove any CRLF and replace it with single LF, which can mess up the whole image.

To force Git to consider a file binary which it would consider as text-file otherwise, the easiest way is to add a .gitattributes file to the directory containing the file or to any parent directory. In my case, I normally add a .gitattributes file in the root of the repository, containing

*.eps -text -diff
*.jpg -text -diff
*.png -text -diff

In the file you set attributes to a path (or a pattern), or unset them (with the minus sign).  The text attribute is the attribute which tells that end-of-line normalization should be applied to a file. If you unset it, Git won’t mess with the line endings in the file and consider it binary.

More details can be found on the gitattributes man page.

My initial git settings for any repository

This is my cheat sheet for the settings I use for my git-repositories (list to be edited continuously):

Global settings:

git config –global user.name Martin Carpella
git config –global user.email xxx@yyy.invalid
git config –global color.ui auto

Per-repository settings:

git config core.autocrlf true
git config core.safecrlf true

Per-repository .gitignore for Visual Studio/C# projects:

bin
obj
*.user
*.suo
Thumbs.db

git-svn fails with fatal error: unable to remap

Git‘s nice Subversion (SVN) integration is one of the reasons I switched to using it within our company for my own revision control besides our official repository. Unfortunately, upgrading cygwin broke my system once again:

$ git svn dcommit
6 [main] perl 4760 C:\cygwin\bin\perl.exe: *** fatal error - unable to remap C:\cygwin\lib\perl5\site_perl\5.8\cygwin\auto\SVN\_Core\_ Core.dll to same address as parent(0x260000) != 0x990000 84 [main] perl 3224 fork: child 4760 - died waiting for dll loading, errno 11 panic: MUTEX_LOCK (45) [util.c:2331] at /usr/bin/git-svn line 787. panic: MUTEX_LOCK (45) [op.c:352].

The reason behind this behavior is a huge difference in the way processes and threads and libraries are created/handled on Windows and Linux. git-svn relies on perl within cygwin and several perl libraries that use the same base-address for libraries internally. Of course, no two libraries can be loaded to the same base-address at the same time.

Long explanation, short way to fix the problem:

  1. Quit all cygwin processes
  2. Start ash (<cygroot>\bin\ash.exe) (<edit>Use “Run as Administrator…”</edit>)
  3. Execute /usr/bin/rebaseall

Voilla, that’s all. git-svn should work again.

.NET strings are not always immutable!

Strings are immutable. If you want to modify a sequence of characters, use StringBuilder. At least, that’s whats officially said. But in the framework there is at least one method that does modify a string:

TextRenderer.MeasureText() with ModifyString and EndEllipses will modify your string to match the ellipsed text if ellipsing happens. You can look at this VB# example on codeproject using TextRenderer.MeasureText() for trimming text on how it is used.

The string seems to be modified directly in native code by DrawTextEx from user32.dll. Additionally to the scary fact that strings are not immutable, the length of the string is not updated, regardless if the resulting string is shorter!

For instance if you have a string “aaaaaaa” which will be truncated to “aa...“, the Length property will still return 7 for the shortened string. The debugger shows that the string will in fact be “aa…\0a” after the operation. So maybe it might be right that the string is still 7 characters long but most outputting functionality like Console.Out.WriteLine() gets confused sometimes and stops any further output to the debugger or console under certain conditions.

A very quick investigation of the System.Drawing assembly using Lutz Roeder’s fabulous .NET Reflector showed that at least there should be no memory corruption in case “WW” would get ellipsed to “W...“, as DrawTextEx takes the length of the buffer and should result only in “W.“.

Summing up, I find the corruption of an immutable string by an official Microsoft API very troubling.

Content-aware image resizing

Krispin made me aware of a very cool new technique for resizing images: content-aware image resizing. Based on an energy-function path of an image are removed when shrinking or are duplicated and interpolated when growing the image in a non-uniform way.

This technique can also be used to remove objects from a given image. There is a nice demo video available on YouTube (it’s the same as in jfo’s coding blog, where Krispin originally found the information):

[youtube vIFCV2spKtg]

(via jfo’s coding and slashdot)

git-svn on Windows (cygwin)

Update 2008-10-10: Often perl will not work due to memory-remapping problems. A solution can be found in my article about the issue.

What I really love about Git is the fact that it nicely integrates with existing Subversion repositories. At our company, we are using Subversion as our SCM, but I personally like Git more and I want to use it as a side tool for more flexible branching, merging, and for checking in versions I wouldn’t check in the shared repository.

Git is supplied with git-svn, which can import an existing SVN repository and also commit back to it. Under cygwin, you need to perform two additional steps for getting git-svn to work, otherwise it is likely to fail with “failed to include Error.pm”.

You need to download Error.pm from CPAN. You have to save it to <cygwin-dir>\lib\perl5\Error.pm

Voila! git-svn should work now.