I hold the title of Enterprise Manager for BI and Analytics at McGrath RentCorp. We are a rental company with one manufacturing and four rental divisions. The company has been in business for over 40 years, with operations throughout the United States, and in Canada and a few other countries. My team provides information to internal customers through different BI and data warehouse platforms. My responsibility is to manage the BI and Analytics team and service internal customer needs.

I have been working in the BI and Analytics space for 14 years. However, I began my career in web development. I worked at Tata Interactive Systems in Mumbai, India before coming to the United States in 2000. When I first came to the Bay Area, I was involved in several contractual engagements working on web applications. In 2001, I received an opportunity to work with McGrath RentCorp when the company sought to create its web presence with a site called eRentTestEquipment.com. Before the site went live, the company hired several contractors to carry out certain enhancements and bug fixes. That was my first role at McGrath RentCorp. In 2007-08, the company decided to implement a sophisticated rental and logistics engine called Rental Result for one of our largest divisions. This logistics engine came with the Cognos BI platform, which had not yet been acquired by IBM, and I was offered the opportunity to switch gears and begin working on BI.

Making the shift involved a definite learning curve. While the tool itself was not difficult to gain expertise with, the model, platform, and the data being served through the Cognos platform required greater understanding. This is primarily what I have noticed with other platforms as well. From a self-serve perspective, the tools are not difficult to use. However, people struggle with developing an understanding of the data, model, and framework.

One of the most important lessons I have learned is regarding the acquisition of BI products. A few years ago, McGrath worked with a mid-sized BI company that bundled it’s reporting tool with Rental Results and we were offered limited options to evaluate different BI products. This continued in 2012, when we implemented the business applications from a major software provider. Again, we did not go through an evaluation process of the various tools or BI platforms available at that time, in terms of the best fit for our environment and operations. Organizations may think of themselves as an Oracle shop or a Microsoft shop, and pick their BI platform accordingly. However, it is important to evaluate different products rather than picking just one technology provider.

We are now undertaking a transition to a new BI tech stack. For evaluating this new stack, we came up with a matrix of key criteria to consider, identifying reducing complexity, reducing cost, and increasing performance as the primary goals. When shifting from a homegrown, on-prem setup to the cloud, organizations will certainly want to increase performance to be able to scale further. However, that has to be balanced with the immediate costs, when you have a smaller user base, for instance. Simply put, organizations should classify the various problems they want to solve using a new technology, rate the different available platforms in terms of pricing, complexity, overheads, performance, and other factors, and then choose their products appropriately.

Next, a proof of concept should be mandatory before implementing an actual product. With cloud platforms offering free credits, organizations should leverage them to test products under real conditions. They should deploy their actual data in the PoC and test how a platform performs, and how different it is from other available products and the existing platform.

I manage a mid-sized team, hiring contractors to scale up as needed. Our data is currently warehoused in OBIEE. However, we are planning to transition away from OBIEE in the next three years. Last year we implemented our own custom BI stack, developed after evaluating several modernized analytics platforms. The front-end of this new platform is hosted on Tableau, with Snowflake serving as the data warehouse, and Rivery as the ELT platform. This will help us assimilate data from a variety of sources, including on-prem and direct sources, and from the cloud as well.

The evaluation process was challenging and time consuming overall. Given the reliance on our critical data infrastructure and insights that the business teams were dependent on, we planned the transition over a 3 year period. The transition is currently ongoing, hence I’ll share more about how we went about the process.

The main goals for the transition were reducing complexity and cost. Based on these goals, we shortlisted technologies to modernize our data stack. We created POCs to help evaluate the technologies that we would use for the transition and zoned in on the Tableau / Snowflake / Rivery stack as our final stack.

We decided to go with two parallel (legacy and modern) data stacks to reduce business impact. Once we stood up the new infrastructure, we started transitioning dashboards from the old stack. On completing validations across stacks, we switched off lights on the legacy dashboards.

Our transitioned users were amazed with what they saw and with the options that Tableau enabled for them, notwithstanding the benefits of Tableau Online for getting benefits to incremental features.

Transitioning to Snowflake, a managed service on the cloud was extremely beneficial to our development teams too, reducing the need with mundane activities like backups and like. This opens up new capabilities allowing for better management including creation of new warehouses for Tableau, Workflows, etc. allowing us to manage costs better

With Rivery, we get ELT in cloud, fetch data from cloud sources, much faster than tools like Infomatica. Very competitive in costs from the features delivered. Allows for a large footprint in data structures with * Variety of sources and destinations * New insights delivered to business * Adoption is picking up

We experienced no impact given our transition model. We left the legacy BI stack running and it is still operational and running in parallel. We are retiring content (dashboards) one at a time after re-creating and improving the same in the new platform.

My guidance to teams undertaking this journey would be:

• Pick a platform pay-as-go and scale up
• Keep existing platform running to minimize costs
• Understand and use feature overlap between stacks to your advantage
• Review your lessons learned and factor in learnings while you go through the transition
• Cost of delivering new features on older stack is under represented and needs to be accurately planned
• Snowflake transition - We have had to build everything ground up. Document naming conventions, view and table naming conventions. The more organized and planning effort that you put in the better. Have clear guidelines for data engineering team to come out ahead after the transition.

The Benefits of the cloud are tremendous. Firstly, you don’t need a DBA, you don’t need admins to monitor backups and the up time and speed is incredible. You pay only for what you use and slowly scale up when you grow. You also don’t need to worry about patching and upgrading to a newer version. Only downside I see is the flexibility to customize certain things buts that a small tradeoff.

• Data discovery used to take 75% time and 25% to uncover insights, now flipped
• Less resources/cost of managing tech stacks and keeping the lights on
• Can focus more on the business
• Increased performance for the end users
• In the past they didn’t have the patience for dashboards to load and requested email, now performance has flipped the switch

I feel that the term BI is itself dated, in that the space has expanded to include predictive analytics and insights. In my mind, business intelligence involves serving information from existing datasets, while analytics and insights involve uncovering new or undiscovered areas that could drive your business. I would also add that artificial intelligence, machine learning and augmenting external data are no longer nice-to-haves but are slowly becoming must-haves to ensure a competitive advantage in the data space.

The most exciting aspect is the range of data sources we get to work with. With application teams, applications do not change on a regular basis. We are constantly in the process of extracting data from new sources as well as augmenting data from traditional sources. So, there is always the excitement of dealing with new sets of data and uncovering new insights for the business.

My primary challenge comes from managing a small team that is simultaneously responsible for production support—managing tickets, change requests, and so on—and for working on specific data projects. Even when contractors are hired for more complex projects, our internal sources become involved because of their business and domain knowledge. This is exciting for my team members. However, they do get involved into the production support aspects more than they would like. It has been challenging to balance these two aspects, but we still have to keep the lights on with the routine work of production support.

When hiring a developer, managers often tend to focus on technical interviews. However, nowadays, a developer can get up to speed on new technologies in no time with the range of resources available online. So, hirers should not be rejecting candidates on the basis of experience with a particular technology. They should focus on the overall experience of candidates, giving them an opportunity to get up to speed and gain knowledge of particular languages with which they are not familiar. I would rather look at the candidate’s prior work and experience, rather than focus on a particular skill set in terms of specific programming languages.

If a candidate has switched between too many projects in the recent past, that is a big red flag. In terms of BI and analytics, getting up to speed on the data itself requires between three to six months. So, I prefer candidates with a minimum tenure of at least one-and-a-half to two years with each company.

On the plus side, one thing I look for is how the work of a candidate at previous companies helped drive the business forward. BI and analytics have evolved significantly in the recent past and are no longer about merely serving information. They now involve developing insights and using analytics and insights to change and drive business, generate more revenue, cut costs, and so forth. I want to see information on how a candidate made a difference to a company’s business in these directions.

For example, we recently developed a sales rep analytics dashboard on Tableau and Snowflake for one of our business units. A previous version of this dashboard existed on OBIEE. The OBIEE dashboard was more tabular, a pivot-type that was interactive to a certain extent and allowed us to filter data using different prompts and drilling through at one level. However, using the power of Tableau, we were able to uncover several new insights into sales rep performance. In the tabular format, it was difficult to clearly capture several metrics such as month-to-date pacing on goals, the conversion ratio on outbound calls, leads, quotes and bookings. Using colours, graphs, and charts in Tableau, we have been able to uncover several insights regarding our top customers, where we are losing and gaining business, which segments to focus on, and so forth.

Candidates should be prepared to talk about their prior experience and assignments in terms of how they were able to resolve or answer business questions. Moving forward in analytics, it is not about receiving requirements from business and delivering insights as is. Instead, BI teams must actually ask businesses what questions they would like to answer using analytics and review how well their analytics can answer those questions. So, candidates coming in for an interview should be able to give examples of how they have solved business problems by answering the questions raised by business. That would showcase their work and their skill sets in a unique way.

I think the first step is for young candidates to figure out their interest, whether they are even interested in dealing with data and information. Beyond this, the subject itself is very wide. So, young developers and analysts could become proficient in front-end tools like Tableau if they are more interested in the art of presenting data. If they are more interested in the technical aspects and want to deal with questions of business logic, queries and so forth, their focus would be on the back-end, the data warehouse and the transformation of data. Finally, candidates should focus on some particular technologies that are most popularly featured in job requirements.

We are currently building up a next-generation system for one of our largest divisions. We are also going to be deploying next-generation customer relationship management, CPQ (configure, price, quote), and so on. BI plays a major role in these developments. New systems mean new dashboards, new insights and so on.

At McGrath RentCorp, we do not have a data volume issue. However, we do face a velocity issue. We deal with a variety of disparate data sources as we serve four rental divisions using a range of different applications. So, our work involves transforming and marrying a lot of data on a regular basis.