The conversation that changed the way I looked at writing software programs

When Santa comes to town, we get goodies, when a Wise man visits, we get salvation.
Living in Singapore changed you forever, you will never be the same and will never see things the same way again. I mentioned this to a friend who lived abroad, working for a renowned Airline company in the world.

He paid a visit to my hometown after his probation and we were having dinner together with friends near my college campus. While we were quickly checking through the menu, he started chatting about the future in IT industry and what it would look like. He was very enthusiastic about narrating his experience in the office and how he was pleased by some of the experiences he had in the office.

As we were passing around a plate of roasted Crab, he gladly began explaining what life was like for him in Singapore. He told us about the work, culture, experiences, and opportunities that he was exposed to abroad. As we switched our discussion to technical topics, he started talking about a programmer in his office who write 300+ lines of COBOL code in a day, compile it once, push the binary for execution and go home. The team was using IBM mainframe systems and in those days these systems are being used more selectively and efficiently as each minute of utilization will cost money. My friend was very much amazed to see how someone can write code that compiles successfully in the first attempt and run without any glitch.

At the end of the conversation, when everyone shuffled out for the night and mumbled their goodnight’s, I couldn’t sleep, I kept thinking about this man who can write the code without any bug in the first attempt and able to run the batch program without any tester. Instantly he becomes my role model.

I have seen project teams who find solace in 90/10 rule of project management where the last 10% takes 90% of the time. They always quote this to justify project overrun. The real issue is that developers who are backed up by one or more quality assurance (QA) testers don’t fully test their code and the QA never put effort to fully cover the scenarios. The responsibility is split between them. It’s vague. The developer is busy producing new features, to finish the first 90%, but only at the last 10%, we get the system to the actual users and it doesn’t work.

There is merit in insisting developer-based testing in software projects, ask the developers to produce automated, integrated, code-based tests and measure the code coverage. Only when the developer understands the user scenarios well, he can do the job right. I would like them to cultivate the habit of writing code that works. I am not saying we shouldn’t have testers, but they are not paid for pointing developer mistakes and programming bugs.

Today I am on the verge of completing my 20th year In the IT industry. I have executed many different projects of varying sizes. I always believed in developer-based testing in our projects and the results are amazing, with no obvious bugs, no cost overrun and better customer satisfaction and we never had QA’s in our team.

Winning Customer

Customers are our business assets, no one disagrees with it, but winning a customer is not just about winning their business, it’s about winning their heart. Business is all about faith & trust, we must earn our way to a good relationship with customers. For any successful business there are long term vision, sustainability & profits, if you reverse the order, it’s only counterproductive.

Most of the startup or SME companies are focused heavily on acquiring new customers. It’s not a bad thing to acquire new customers, a startup has to acquire new customers to become a viable business, but as the company grows in size, it becomes imperative that the lifetime value of its customers should be improved, it must be significantly more than their acquisition cost. If you are dealing with too many new customers every year, you are not only draining the company resources to onboard and gain the trust of the customer, it will also impact the company’s brand in the market and employee morale.

To win over a customer, we need to master three essential components. A sound psychologya logical solution, and an effective risk management plan. These are like 3 legs of a stool, you lose one, you will fall. There is no shortcut to this, people want to believe they were smart, they over-promise, provide unviable solutions and do not focus on the business benefits. Even though their smartness brings initial successes, it is short-lived and wouldn’t add any value in the long run. Impulsive actions, taking chances and those who feel that the customer owe them the business are losers who try to create an omnipotent feeling of bliss than realistic long-term success and are plagued by self-doubt.

It must start with what the customer wants and define all the actions based on clearly defined rules. You must structure your risk management plan so that the initial teething problems wouldn’t kick you out of the game. As you go past the honeymoon period, there must be a sound customer retention strategy to improve satisfaction and profits. For example, one study showed that a 5% increase in customer retention rate increases profits by 25% to 85%, but most of the time this aspect is overlooked by the management and their sole focus is to acquire more and more new customers. Sad to say, most of the companies I have seen do not have a solid Customer Life Time Value framework (CLTV). They have very little appetite to spend on improving its customer retention rates, customer satisfaction or customer relationship.

The concept of the long-term value of customers and the value of relationships is not new, but its not a core strategy in many of the IT services companies. Nobody wants to lose one of their largest, most strategic customers, but if you’re not investing in them properly and/or not providing the value they need, there is a real possibility that they will dump you, sooner than later.

No Code is better than “No Code”

I have a collection of technical articles that I put together in 2005-2008 period. I want to publish them through my blog. This particular one Inspire me a lot and the emergence of Low Code tools only reaffirm its relevance. An excerpt of the article that is published in the smart-techie magazine(2008 edition)

Higher wage costs in India forces IT services organizations to look at innovative methods to run the floor to deliver value. The uncertainty and fears may be rising, but there is a prospect of bringing back the old shine. The organizations find comfort in following what they were doing for long to build software, but in the changing scenario, it will be difficult to ensure business success unless organizations become more agile to adopt new methods to build software that will eventually address the people and cost factor associated with it.

There is No Code that is better than “NO Code“. Each time you increase the amount of code by adding more programmers, your software grows exponentially and more complicated. It also adds more overheads in terms of proportionate quality assurance, HR and so on. More time & money should be invested in identifying new methods to write applications that will help us in achieving high-quality code quickly and help customers lower project cost and time to market. With a unique combination of SDLC Boosters based on MDA (Metadata- driven architecture) and TSPC (Technical Statistical Process Control) companies would be able to reduce the overall project effort and improve productivity by 40 percent in building Business Applications.

When the traditional model dedicates 40 percent time for programming and another 20 percent for Q.A. metadata-driven approach demands only 20 percent in coding and the team spends more time on design and analysis.

Moving to a Metadata-driven development means that an organization will probably implement significant changes to their software development practices. But it’s worth the effort. One of the daunting issues of traditional practices is more programming and less of designing. A fully Metadata-driven approach means that the output of analysis and design goes straight into the software and there is not much code written. Therefore, moving to a Metadata-driven approach can provide a major benefit in terms of software quality and speed of delivery to customers. When thinking about executing models, most people inherently think about two options. One is code generation and another interpretation. Code generation is preferable since it is much simpler than interpretation and it is more testable and resulting code can be inspected. Whereas an interpreter is a (meta-) program that reads the model and executes code (algorithms, UI rendering) as it queries or traverses the model

With the changing business scenario, the usage of high-level abstractions in the solution specification makes great sense. We foresee the industry adopting DSL in driving quicker and better value to customers. DSL’s are poised to take on many of the challenges we have today, including the frequent requirement changes.

Will JavaScript lose its sheen??

JavaScript is an interesting programming language, it is the One “Programming language” for everything.  It runs natively in the browser, it is the only predominant language that is interpreted and executed by all browsers.

JavaScript has emerged as a powerful server-side language (Node JS). There are millions of MicroServices written on NodeJS. They are many server-side frameworks in JS that replace traditional server platforms and desirably it is the most used programming language on earth.

JS is versatile, it has the lowest threshold to get started, it is full-stack, it is omnipotent, the list goes on and on and there is no stopping for JS. Knowing JS is rewarding, It is JS, that’s born to rule the internet.

 Wait… that is the problem, we are talking about a single entity dominating the webspace, we can’t have anything but JS on the web, we are accepting the hegemony. No…… Not at all. Welcome to WebAssembly. JavaScript is not suitable for everything. Its performance on browsers is not great for all type applications (if JS had everything then why did people used Flash and Silverlight’s of the world). 

Web-Assembly is a new set of specifications that are bringing near-native performance to the browser. Microsoft is keenly working on a compiler that compiles C# to Web-Assembly. It will be interesting to see if JavaScript will lose its sheen in the process and will make way for newer paradigms like Web-assemblies.

Essentially it is humanity’s quest for improvement that makes our living interesting.

Don’t let your customer uncover the errors

The Boeing 737 Max has been in the news because of two crashes, practically back to back and involving brand new airplanes. The 737 Max was designed to fly like the popular 737. This was an attractive proposition for the company because they need not have to go to safety regulators for new norms. The new 737 Max needed a larger engine, the over-sized engines required a structural change to the airframe of the plane. But structural changes are expensive. Hence for economic reasons Boeing decided to fix it in the software way

What is “software way of fixing”.

In 1994, the iconic Pentium processors were affected by what was known as the FDIV bug. It affected only a tiny fraction of Pentium users. But the effects on the company were quite pricey. In 1994 Intel recalled the (slightly) defective processors and it did cost the company $475 million, a significant portion of the net income in those years.

While Intel was struggling to match up to the investor’s maths, the other company was riding the tide. That company was none other than Microsoft. You may recollect the biggest embarrassment Bill gates had once.

Microsoft chairman was demonstrating the plug and play features of their new operating system to a group of influential people, very large audience,pin-drop silence, everyone was excited and looking up to bill gates for his maverick act, but all he got was Blue Screen of Death (BSoD), and a “whoah…”. 

Windows had major bugs, but Microsoft addressed these errors with periodic software updates, that cost them nothing compared to Intel. But it created a cultural change in the way people look at errors. Even when people know that things are not complete when they leave their premises, they care nothing. Because they realized they didn’t have to be complete. People are not perfect so is my product. The job could be done at any time in the future with an update. But this created cultural laziness within the community.

They are following “shoot first, ask questions later” philosophy.

Boeing tried to copy this philosophy. They knew that 737 Max required a serious investment to make it happen, but they let it go until the error is uncovered by their customers. They relied on probability theory, optimism, and GOD. But at the end of the day, they have lost more money and most importantly the trust of their customers.

Ironically I have experienced this in my career. I relied upon “shoot first” theory for one of our customers in the past and it backfired. Luckily I was not building an airplane, hence the damage was contained. But I may not be lucky next time.

Are you “The Lion Leader”

Leadership is an important aspect of a business for growth because leaders know the way, show the way and goes the way. They develop strategies that give your business a competitive edge and grow the profits. Great resources regarding leadership can be found in the animal kingdom. 

The primary example is the Lion. Lions are the only cats that live in groups, which are called pride. Lions are called King of the Jungle. He is a symbol of power, strong leadership and authority. Though they are powerful and promote community living, Lion leaders are not necessarily an asset to an organization. Here is why!

  1. The Alpha Lion, do very little job themselves. They delegate work to others.
  2. The Alpha Lion kills the cub when they switch the Pride. Like new leaders firing existing employees and take control over.
  3. The Alpha Lion never protect others, they protect their ego. They fight for dominance.
  4. The Alpha Lion takes the lion share of the lunch. Too bad, the lioness kills the prey, but the Alpha lion takes the Liver. Not willing to share the success with the deserving.
  5. They are lazy, sleep 20 hours a day. But when wake-up, they roar loud to frighten others and to get noticed.
  6. They also face the worst end of their career. Full of hardship, starvation, and nomadic migration. They are deserted by their pride and left alone to be eaten by Hyenas. Now tell me, do you take pride in being a Lion Leader??

Office 365 Throttling -How to handle.

Office 365 SharePoint Online enforces throttling limits to prevent users from over-consuming resources. When a user runs CSOM or REST code that exceeds usage limits, SharePoint Online throttles any further request from the user for a brief period.

Find out about throttling in SharePoint Online, and learn how to avoid being throttled or blocked. Includes sample CSOM code you can use to make your life easier.

What happens when you get throttled in SharePoint Online?

When a user exceeds usage limits, SharePoint Online throttles any further requests from that user account for a short period. All user actions are throttled while the throttle is in effect.

  1. For requests that a user performs directly in the browser, SharePoint Online redirects you to the throttling information page, and the requests fail.
  2. For all other requests, including CSOM or REST calls, SharePoint Online returns HTTP status code 429 (“Too many requests”), and the requests fail.

Why throttling is tricky subject

Throttling is depending on the overall traffic at given point in time. This cannot be predicted, especially when the SharePoint farm is shared with multiple customers.


What is the best practice to handle throttling

  • Decorate your calls with user agent info. This will give priority for the calls over other normal calls
 using (ClientContext ctx= new ClientContext(sitename)){
    SecureString passWord = new SecureString();
    foreach (char c in KEY.ToCharArray()) passWord.AppendChar(c);
    clientContext.Credentials = new SharePointOnlineCredentials(UserId, passWord);
    Web web = ctx.Web;
    ctx.ExecutingWebRequest += delegate (object sender, WebRequestEventArgs e)
    {
       e.WebRequestExecutor.WebRequest.UserAgent = "NONISV|<company>|app>/1.0";
    };
 }
  • Implement A Resource Count Semaphore in the program so that it wouldn’t allow more than 30 calls concurrently
  • Implement A Back off algorithm is implemented to detect the throttling and retry after the prescribed time
public static class ClientContextExtension
{
    private static SemaphoreSlim _sem=new SemaphoreSlim(30);

    public static void ExecuteQueryWithIncrementalRetry(this ClientContext clientContext,int retryCount,int backoffInterval)
    {
        try
        {
             _sem.Wait(); //block the call if more than 30 concurrent calls
            int retryAttempts = 0;
            int retryAfterInterval = 0;
            bool retry = false;
            ClientRequestWrapper wrapper = null;
            // Do while retry attempt is less than retry count
            while (retryAttempts < retryCount)
            {
                try
                {
                    if (!retry)
                    {
                        clientContext.ExecuteQuery();
                        return;
                    }
                    else
                    {
                        // retry the previous request
                        if (wrapper != null && wrapper.Value != null)
                        {
                            clientContext.RetryQuery(wrapper.Value);
                            return;
                        }
                    }
                }
                catch (WebException ex)
                {
                    var response = ex.Response as HttpWebResponse;
                    // Check if request was throttled - http status code 429
                    // Check is request failed due to server unavailable 
                    //- http status code 503
                    if (response != null && (response.StatusCode == (HttpStatusCode)429 
                            || response.StatusCode == (HttpStatusCode)503))
                    {
                        wrapper = (ClientRequestWrapper)ex.Data["ClientRequest"];
                        if(wrapper == null)
                        {
                            logger.Info("CSOM Wrapper is null");
                            throw;
                        }
                        retry = true;

                        // Determine the retry after value - 
                        //use the retry-after header when available
                        string retryAfterHeader = response.GetResponseHeader("Retry-After");
                        if (!string.IsNullOrEmpty(retryAfterHeader))
                        {
                            if (!Int32.TryParse(retryAfterHeader, out retryAfterInterval))
                            {
                                retryAfterInterval = backoffInterval;
                            }
                        }
                        else
                        {
                            retryAfterInterval = backoffInterval;
                        }
                        // Delay for the requested seconds
                        Thread.Sleep(retryAfterInterval * 1000);

                        // Increase counters
                        retryAttempts++;
                        backoffInterval = backoffInterval * 2;
                    }
                    else
                    {
                        throw;
                    }
                }
            }
            throw new MaximumRetryAttemptedException(
                    $"Maximum retry attempts {retryCount}, has be attempted."
                    );
        }
        catch
        {
            throw;
        }
        finally
        {
            _sem.Release(); //release the semaphore
        }
    }

    public class MaximumRetryAttemptedException : Exception
    {
        public MaximumRetryAttemptedException(string message)
        : base(message)
        {
        }
    }
}

Now call the extension method as below in your CSOM code.

clientContext.ExecuteQueryWithIncrementalRetry(3,5000);