Mert Emin Kalender

Best Practices Of Deploying Windows Services To Azure

15 July 2015

Azure provides a set of services and components that you can use to develop distributable and scalable applications. Even though the whole platform and its pieces (names, versions, definitions etc.) change every so often, the value you can get from it is still high. However, the lack of documentation creates an ambigiuty about how things work or what is possible and not. Often I found myself digging through nth search result or years old blog to figure things out. That’s why I want to share what I have learned about deploying Windows services to Azure.

Background

Recently I had a task to create custom out-of-process diagnostics and monitoring agent for cloud roles. I thought that was going to be an easy task, since I could just create another role and implement the agent within the role. Using Azure role definition entry points for start, stop and update, it was going to be pretty clear and simple implementation.

I was pushing log entries to ETW (Event Tracing for Windows) and generating performance data (using System.Diagnostics performance counter definitions) in the actual role. The agent role was able to listen ETW for logs and read performance data periodically. It didn’t take long till I figure out roles are like virtual machine templates, which means each role goes to a separate virtual machine, by deploying my locally perfectly working proof of concept.

After some research, I learned about startup tasks. I converted my agent role to a Windows service and created a startup task to start up the agent service when role is being started on the cloud. In the following, there is a quick summary on how to deploy Windows services along with a role.

Create the Service

Create a new Windows Service project. The project will come with the following files:

Add a Service Installer

Add a class to install service to the machine. First, reference System.Configuration.Install library and then add a class, let’s call it AgentInstaller, which should look like the following code piece. Select your service’s Account type (whether it is going to be Service account or System account) and StartType (manual or automatic). With manual start option you can pass arguments to your service from your cloud role and make the service general to work with given parameters. In my case, I was passing connection strings, table names, app tokens to the agent service.

using System.ComponentModel;
using System.Configuration.Install;
using System.ServiceProcess;

namespace WindowsService1
{
    /// <summary>
    /// This the installer for the monitoring service.
    /// The monitoring service is installed to the target machine as a long-running local service.
    /// </summary>
    [RunInstaller(true)]
    public class AgentInstaller : Installer
    {
        /// <summary>
        /// Process installer
        /// </summary>
        public ServiceProcessInstaller ServiceProcessInstaller { get; set; }

        /// <summary>
        /// Default constructor
        /// </summary>
        public AgentInstaller()
        {
            ServiceProcessInstaller = new ServiceProcessInstaller { Account = ServiceAccount.LocalService };

            var serviceInstaller =
                new ServiceInstaller
                {
                    ServiceName = "Your service name",
                    Description = "Your service description",
                    DisplayName = "Your service's display name",
                    StartType = ServiceStartMode.Manual
                };

            Installers.AddRange(new Installer[] { ServiceProcessInstaller, serviceInstaller });
        }
    }
}

Reference Your Service in Role Project

Find your Web or Worker role project (an MVC project, a class library etc) and reference Windows service project. Make sure to enable copy local option to bring dependent assemblies. If your Windows Service project depends on an external library, be sure to enable copy local there too. This is where you might have issues. You service executable will be placed to the app folder of your role. Therefore, it will be expecting libraries (dlls) to be placed there for successful execution.

Copy Installer Tool and Create an Install Script

You need the installer tool and install script to install the service. The tool is not available in the virtual machine by default. You need to copy the executable yourself. Find it on your machine or download from the Web. Then, place the executable to the folder of your role in your cloud project’s (not your role project). Here is a post on that. You can do this by dragging and dropping the file onto role name in the project. For web roles, create a folder called bin/ first, since Web executables are placed in a folder called bin in application root folder. This copy will make sure the executables are placed in your application’s root folder.

Now create an install script, let’s assume it is called InstallAgent.cmd, in the same folder described above. The sample script given below first stops and deletes if service is running, and then reinstalls the one in the newly deployed package. "%TEMP%" is an environment variable pointing to a temporary folder to create files. Therefore, you can use to write logs that might be useful later to diagnose problems.

:: Stop and delete the current service
sc stop "Your service's name" >> "%TEMP%\StartupLog.txt" 2>&1
sc delete "Your service's name" >> "%TEMP%\StartupLog.txt" 2>&1

:: Uninstall and install the service
InstallUtil.exe /u "Your executable name" >> "%TEMP%\StartupLog.txt" 2>&1
InstallUtil.exe "Your executable name" >> "%TEMP%\StartupLog.txt" 2>&1

:: Exit
EXIT /B 0

Create a Startup Task

Create a startup task in your service definition file. Select the execution context and task type. This will initiate a task with given script file when a new package is deployed.

<Startup>
	<Task commandLine="InstallAgent.cmd" executionContext="elevated" taskType="background" />
</Startup>

Create a Startup Task

Set the execution context to elevated your service definition file.

<Runtime executionContext="elevated" />

Start Service in Your Role

Start your service in your role’s OnStart entry point using ServiceController helper classes.

var agentController = new ServiceController("Your service name");
try
{
    if (agentController.Status != ServiceControllerStatus.Running)
    {
        agentController.Start(
            new[] { "argument 1", "argument 2", "argument 3" });
        agentController.WaitForStatus(ServiceControllerStatus.Running);
    }
}
catch (Exception exception)
{
    throw;
}

Every Once In A While

27 December 2013

We should sit and think about how things are going with our careers. We should ask at least some of the following questions to ourselves: How are we doing lately? What’s wrong? What’s not? What’s missing? What’s next? What I’ve learned? What I’ve failed to learn?

Questions like these should help finding out the current situation in essence and deciding how to make things better in the future and creating an action plan. I think most people do create this action plan at some point intentionally or not. Although it is not always realistic or practical to do it frequently, there may come some time when an action plan becomes inevitable. I guess New Year is one those popular times, since it is so common to hear all these “New Year’s resolutions” at this time of the year. However, I find it useful to do it when there is a significant change in my career or my career goals.

The following is an half-done review of recent time frame and action plan for myself. It may seem like another “resolution”, but it is not.

What I’ve learned:

What I am going to do:

Finally, I think nothing/nobody is perfect, but you have got to try to achieve the best you can.

Do We Really Connected With Each Other

18 March 2013

Today is one my of Facebook friends birthday. All day people congratulated him for the day of his birth. However, there is one problem. This person died last year, and only a few people are aware of that fact. The rest majority acted without a clue as if he is still alive.

Facebook’s entry page says that we can connect with friends and the world around us using it. But how come two people are connected with each other, become friends, and one is not aware of the fact that the other is death? This is sad, really sad showing an aspect of damaged social interactions happening online. For me all these congratulations, condolences, best-wishes are prefabricated. Degrading the whole social in-person interaction to a numb but decorated with smileys, sad faces statement does not seem genuine. So I mostly avoid these, although this may not be the best solution.

I think social media should be an alternative way to interact with people, but not the only way. I like the way social media enabling more sharing with others and expressing someone himself/herself with a text or song or video. However, I really don’t like the fake social interactions not to mention the online profiles, which has nothing to do with the real personality.

UPDATE

Here is a great article about sharing online. Not totally related, but worth to read.

Personal Resilience Engineering Learning To Recover

30 January 2013

The other day I was reading an article titled Resilience Engineering: Learning to Embrace Failure on ACM Communications. The article is all about accepting that system failures exist and they are inevitable. All we need to do is preparing for them rather than trying to prevent. In the article, the best practices of resilience engineering at Amazon, Google and the other companies are told by system engineers. But how about resilience engineering for personal failures? How should we deal with failures in our personal life?

Failures are real and they happen more frequently than expected. The range of failures in personal level is huge: from school to work, health to relationships etc. You can add as many as you want. Getting the job you wanted, passing a course, finishing up a project or even completing some km on treadmill to lose some weight may or may not be possible from time to time. This should be totally fine, because having a life with full series of successes seems not likely, but failing seems rather easy.

However, don’t underestimate failures. Failures are good. They tell us what’s wrong and what should be fixed not like successes. If you use the chance of recovering provided by failures, I bet succeeding would be easier and a lot more assuring. Personally I have one simple and one big example on this issue.

The simple one is about exams and may occur often. Whenever I get a low grade on a midterm, I take this as a good sign. I check my paper, try to understand what went wrong. I reevaluate my knowledge on the topic, if necessary my study plan too. Then, I try to recover from this simple failure using the failure itself as a roadmap to success.

The big one was life changing for me. Last year I was so believed in pursuing PhD abroad and becoming an academic, but I failed at the very first step1, that is admissions. Then, I took as much time as I need and went through all my plans. It lasted for almost six months, and I made some huge decisions at the end. The good part is these decisions are now turning out to be right call, and that is what I call learning to recover.

So, I think successes are good, but failures are better as long as you use them.

  1. I wrote a blog post about this experience. ↩

Necessity Of Excessive Content Filtering

25 November 2012

A few months ago I started to follow articles on computer science related issues closely. I first discovered Hacker News. After a while I find out a number of people who writes and shares interesting stuff about technology, academia and life hacks. However, I find myself suffocated in dozens of articles, news stories and tweets soon and I recognised the necessity of excessive filtering of the constant flow of content.

At first, I was amazed with the quantity of content I am exposed to. But this didn’t seem to be a problem, since I was really passionate about reading all of them. Rather than using twitter to read random thoughts of friends and celebrities, I began following some tech related accounts. Twice a day I went over recent tweets. If I didn’t have time to read the article, I added it to my reading list on instapaper. Whenever I got a chance, I opened my reading list and read them one by one. At the same time I continued to discover new people and followed them as well. Before long I realized my consuming rate is not fast enough compared to the incoming rate of articles. My reading list was becoming larger and larger. Suddenly my passion turned into anxiety.

While I was struggling with content flood, I found some other people grumbling like me. James Hague complains about burden of technology and programming content by referring to the problem as background noise in his article. He shares a no-longer-bother-with list of content and states the outcome of his filtering action:

Being much more selective has kept me excited and optimistic and aware of possibilities instead of living down below in a world of endless detail and indecision and craning my neck to see what’s going on above the surface.

Similarly Joshua Gross refers to this issue in The Future is not Real-Time by saying

Content should always feel like a gift, not a burden. To turn it into a gift, we need to start focusing on ways to control the flow. As a result, after realising the enormous flow of content I have been going through, I started to filter out irrelevant ones in terms of my interests. While I check out recent tweets, if title seems interesting, I click and read the linked article. Otherwise, I may still the click and skim through the article and read if it seems interesting. Here it is really hard to define interestingness.

Personally I think an article is interesting enough if I might gain at least some information or insight about anything relevant to my life and profession. Still this is difficult to decide beforehand, but it keeps me away reading long lasting discussions about companies/products/programming languages or articles about internal workings of a startup or some event/incident/news related to a particular geography etc.

In my experience, if you are crushed with a mass of content, what you need to do is

Finally, here a list of twitter accounts I follow:

Best Practices To Screw Up Grad School Applications

02 November 2012

It is fairly easy to find hundreds of posts, pages, lists as in the form of grad application advices, because application process is hard, time-consuming and complex. You need to spend quite considerable amount of your time and money, and make plenty of important decisions. If you do not pay enough attention, the failure is inevitable. This post describes the best practices, i.e. my experiences, leading to a failure in grad applications.

Applying to grad schools is a natural step to take if you want to be an academic or excel at a particular subject in an academic institution. At least this was the way I see the whole process. After I completed junior year of bachelor degree, I decided to apply to the grad schools in the U.S. I failed to be admitted by a good school because of the reasons I come up in the following section.

Before Applications

I failed, because I had insufficient experience in my area of interest. You may think that expecting a fine experience in a particular area for a senior student is too much to ask, but do not forget that there are other candidates with many projects or even publications. I read that some students start working on projects and collaborating with professors from their freshman year.

Choosing Schools

I searched the term “university rankings” on the Internet, and found a bunch of lists. I created an excel sheet and listed universities with respect to their rankings in three of lists that seem to be more acceptable. As a result, I got my own ranking list of universities.

I went over my list from time to time. I checked up each university’s website and looked for particular research groups and/or professors working in the area that I am interested in. I created a generic email with a few variable statements, and sent it the professors whose areas of interest are similar to mine. It was a real surprise to get responses from some of these professors, since I read that they are generally unresponsive to these emails. This is even clearly stated on the their webpages. While some of responses suggested that I should apply first, some others demonstrated professors’ intention to work with me.

Eventually with the help of responses from professors I decided to apply one to two schools from top 10 of my list, eight to ten from 11-50, and two to three from 51-100. I was a bit generous to have that long list, but my idea was to have a tolerant list with a few best shots, mostly fair choices and some worst-case scenarios.

I failed, because I paid more than enough attention to university rankings rather than finding a convenient school/professor match according to the area of interest. I softly went over professors’ research, and then emailed them telling my desire to work with them. By getting a few responses from professors suddenly I began to think that I almost succeed in grad applications. I didn’t know anything about other candidates or financial support or any other related known/unknown facts that might affect the final decisions. I chose to apply to the schools that I had no real idea (these schools were supposed to be worst-case scenarios for me). I thought that they would admit me anyway, but you what they don’t.

Getting Reference Letters

After I got my school list shaped, I went some of my professors and asked them to write recommendations for me. However, I didn’t omit asking about their opinions about school choices, and suggestions for admissions. One of them made sure me that having a long list of schools would increase the chance to be admitted.

I failed, because I think that getting the right reference letters is a hard and complex issue. I still don’t really know what counts good or not for reference letters. However, one reason for failing might be that I had reference letters from professors whose work is not really related to the area I want (except for one whom I worked on a project), but I believe in that they wrote good letters for me. I did even have a professor working social sciences writing a reference letter.

Tests

I took TOEFL and GRE general test just like other students. TOEFL was fine, so did GRE quantitive and writing parts, but I still remember how I was struggling during GRE verbal test. I had no idea about what they were asking. I didn’t know the meaning of most words. I tried to figure out what sentence means from the context. After I failed to do so, I just picked one answer from the list with no reason. However, the result was surprisingly high for picking random answers.

I failed, because for both tests I got scores that seem to be in borderline. I think getting high scores may not directly lead you to a grad school, but low or moderate scores may result in an early elimination in the process.

Interviews

During late January and beginning of February I got requests from some of professors for an interview. According to my experiences, professors generally ask you about your background in the related area and recent projects. They try to get to know you and decide whether you would be good fit to their research group. If you are an international student, your ability to speak English and express what you think and understand what the other person say, i.e. your communication skills in English, are crucial.

Rather than these usual conversations, you can still expect to experience a chitchat. I remember that during one of the interviews, professor told me about the university, city, living expenses, rental prices and other daily stuff except for research. Finally he asked me whether I got accepted by any school, if any where these are. I behaved honestly and told him my options. Two days later, I got rejected.

I failed, because I didn’t have a strong research statement or idea to propose to the professors. I screwed up in one of the interviews, because the place where I chose to be was awful (but I had no other choice).

If you managed to read all the way here, my advice for you is to decide if academia is right for you. Try to learn what is like to be a grad student or academic (check out Stack Exchange Academia). If you think research is your thing, dug deeper into a subject you are excited about. Collaborate with other people, professors, grad students etc. Get in contact with professors you would like to work with in the future after examining their research in details. Take the tests, get high scores. Do not screw up interviews. Be aware of thegradcafe.

As a result, I failed so bad. However, I am happy I failed, because I learned, and gained an invaluable life-experience.

You And Successful Others

22 October 2012

It is quite often that I find myself inadequate about various areas or topics or any piece of work you can think of. This happens when I see someone else who had achieved great things in their life and compare them with myself.

If you have a profession related with computer science, it is easy to find hundreds of people have done something nice. The example must not be a billionaire like Mark Zuckerberg, but an entrepreneur with an idea solving a problem, a developer created a cool application, a grad student written a good paper or even an high school kid sharing his open source code on github are all simple, but significant examples. And I realized that the more chance the person is someone similar to you (e.g. web developer, student), the more you are being hard on yourself. In addition, you can easily find these examples on the Internet by just visiting github, twitter etc. or keeping up with recent tech news or checking your friends’ shared posts.

While this behavior really annoys me, I really couldn’t come up with a good reason to explain it. However, yesterday I read an influential blog post by Matt Cheuvront and learnt that the behavior of comparing myself with other good people is usual. What Matt says is

We constantly compare our beginning to someone else’s middle. Our middle to someone else’s end. And when you do that – you’ll find that you’re never, ever satisfied. You’ll never, ever be good enough. You’ll always struggle to celebrate your accomplishments.

Wherever you are in life with whatever you’re doing – you’re going to be ahead of some and behind others. That’s okay. Own where you are. Take time to embrace and celebrate that. Continue moving forward. And never compare your beginning to someone else’s middle.

So I have decided on some rules that might be useful to cope with this situation. The rules are simple, but I think iterating over and over again would contribute to your self confidence and hopefully lead you to accomplish something satisfactory.

By the way, I saw the link of blog post on Joel Gascoigne’s twitter feed. I suggest you to follow him. He writes and shares cool stuff.

The First Entry Of All Time

19 October 2012

This is the first blog entry of my entire life, because I have always stayed away from blogging. It is common that you can find lots of blogs with a ‘Hello World’ post or even no posts at all. I saw people start blogging eagerly, but after a short while they even forget if they have a blog or not.

Despite the fact that there are good examples (you will probably see links to them in the future), I had generally considered bad ones, and resisted myself from blogging. However, until recently I have realized that I need one. I read a lot, and sometimes I need some safe place to keep interesting posts and maybe share my own ideas. Plus, a public blog might even be better for other people to benefit from it. As a result I created this blog, and here we are. I hope this will be fun!