Set system proxy with a shell script in MacOS

Have you experienced setting a proxy in your MacOS? If you have, you must be fairly clear about how painful it is. You must

1. Open “System Preference”,
2. Find “Network”,
3. Find the right network interface, click on it,
4. Click “Advanced..” in the right panel,
5. Switch to “Proxies” tab,
6. Check the desired proxies,
7. Enter the proxy IP address and port,
8. Click “OK”,
9. Click “Apply”,

and finally, if you are still alive, the task is done.

But, if you are a coder, please do not do that for our title’s sake - you can save a ton of time with the networksetup command. Among a variety of the supported operations on its manual page, -setwebproxy option is for this case.

networksetup [-setwebproxy networkservice domain portnumber authenticated username password]

So, with the following script added in your $PATH, you can simply set the system proxy by typing the script’s name and press enter in your favorite bash interface.

host=10.0.0.10
port=80
networksetup -setwebproxy Wi-Fi $host $port off
networksetup -setsecurewebproxy Wi-Fi $host $port off

🚀🚀🚀🚀🚀

What? You want to turn the proxy off and on in this way? Dig into the manual and see if it bites. 🍺

Move docker files to a new disk

As more and more containers start running in your docker swarm, you may encounter the following exception.

failed to register layer: Error processing tar file(exit status 1): open *** : no space left on device.

As the error message says itself, the disk on your server is full. By default, docker engine stores its downloaded images and running containers in /var/lib/docker, so you can see exactly how much space it has taken with the following command.

# du -h --max-depth=0 /var/lib/docker
9.8G    /var/lib/docker

It’s very easy to purchase an additional data disk and mount to this location if your server is hosted in the cloud, such as AWS EC2 or Aliyun ECS instances.

Firstly, let’s format the new disk.

sudo mkfs.ext4 /dev/vdb1

Then, backup data in /var/lib/docker, mount this disk here and move the data back.

sudo service docker stop
sudo mv /var/lib/docker /var/lib/docker_data

sudo mkdir /var/lib/docker
sudo mount -t ext4 /dev/vdb1 /var/lib/docker

sudo mv /var/lib/docker_data/* /var/lib/docker/

sudo service docker start
df

In my experience, moving data from the backup folder to the new disk took a long time, so it’d better be done in the first place when you are installing the docker engine.

Aliyun MNS dotnet core SDK

It is known that Aliyun has a really bad support for .Net world. Every one of its products and services has a well built Java SDK, but not everyone has a .Net one. Even if some of them do provide .Net SDK, some do not provide full functionality, some do not work in a .Net way and some are buggy and lack of maintenance. Even today, after .Net core has been released for over two years, there is not one product or service officially provides .Net core SDK. This is really bad, especially for .Net prospectors like me.

So, I decided to invent the wheel for those who share the same pain. mns-netcore is then created and open sourced. It now fully supports all Aliyun MNS’ queue operations and still in optimization. This lib is a .Net standard library, which means it works both in .Net Framework and .Net core. To use it, you can reference from Nuget, and meanwhile, you are welcome to raise an issue or contribute by sending pull requests.

In its README, there is detailed specification about how to send and subscribe messages. Finally, please feel free to ask any questions about it on GitHub issues.

How to create a windows service in dotnet core

A couple of weeks ago, we started a new project which is to consume messages from EventHub in Microsoft Azure, process these messages, and store extracted data into our database in Aliyun. Since our team was considering the possibility of using dotnet core to make our apps capable of running on multiple platforms at the time, we decided to take a dive. However, while everything seemed ready, we still had a few troubles to overcome, which include this one: how to run a dotnet core console app as a windows service?

OK, here comes the answer: bridging and wrapping.

Bridging seems more popular. The representatives are these two libraries: DotNetCore.WindowsService and DasMulli.Win32.ServiceUtils. They basically adapt windows service facilities for dotnet core, and everything works just like what you do in .Net framework.

Wrapping is less common but more delicate. In this way, you can wrap a dotnet standard library in a windows service that runs on .Net Framework. This article introduces details about this method. What you need to pay attention to is the compatibility of dotnet standard versions (see the compatibility table here). I have also written a simple app to exemplify this method, you can see it here.

Is that done? Definitely not. The question is actually invalid after scrutiny: as long as you can run an app in the background and somebody can manage its life cycle, why do you need a windows service? The reason why this question came up is that we are still deep in the sea of Windows.

So, although the previous two methods are awesome, my preferred answer is docker. You can put your long-run dotnet core app into a docker image and let the docker engine manage everything. Now, welcome to the ocean of Linux.

Permanently remove files from git history

If you try to search on Google about how to permanently remove a large file that was accidentally committed to your git repository, you may find the following solution.

git filter-branch --prune-empty --index-filter 'git rm -rf --cached --ignore-unmatch extremely-large.file' --tag-name-filter cat -- --all

Unfortunately, this can not help you. The reason is that while all commit history related to the file is completely removed, the git objects are still there. So, if you compare the .git folder’s size before and after, you will find that there is no difference at all, and if you git push -f to your remote repository, these objects are still going with other objects.

A good way to avoid this is to clone a clean repository, change its remote to your remote repository, and push forcibly.

git clone --no-hardlinks file:////path/to/repo repo2
cd repo2
git remote set-url origin https://your-host.com/your-repo.git
git push -f

Now, enjoy your smaller git repo.