A boom starts with a name for it

“Big data” has become a popular term in recent years. Data utilization has been a major challenge for companies for some time. In the advertising business, I was analyzing the impacts on sales by connecting TV audience ratings and scanner data in the 1980s. In addition, in the credit card and distribution industries, data indicating customer trends was being accumulated and used for risk management and other purposes. Looking back on those days, the data was not so substantial, but companies already had years of experience in obtaining information from the data.

What has changed since then is the computing power available and the cost of data storage. Things that were impossible then in terms of technology and cost can now be done cheaply and quickly. We can now utilize a large amount of data that we once had no choice but to discard, and there has been a movement in which companies try to gain knowledge that is useful for their business from such data. I think the current boom in big data originates from this movement.

As is common with any escalation, the big data boom did not emerge suddenly. As soon as the straightforward term “big data” was given to the increasingly larger data sets, it came to the center stage in no time.

New intelligence flows into the marketing world?

I consider that the impact big data has on approaches to marketing research is profound. For instance, companies can easily obtain purchase data in stores and credit card usage history, web access and search histories, and purchase and application records. They can also collect information such as the areas of interest of an individual and their relationships with others from massive amounts of data such as postings to social media like Twitter and Facebook, without using traditional questionnaires or web-based questionnaire surveys.

It is difficult for traditional analytical tools used by market research companies and think tanks to deal with such large amounts of data, and for this reason, some marketing firms have already hired specialists in computer science or physics. According to one version of the story, NASA’s rocket scientists moved to Wall Street upon completion of the Apollo Project. After the collapse of Lehman Brothers, these scientists pursued other areas, one of which was the marketing industry. It is highly probable that new type of players would increase in the marketing world in the future.

A typical recent case may be RTB (real time bidding). RTB is a method of bidding for online advertising. Artificial agents for advertisers place a bid on each page view according to the viewer’s purchase history and behavior traits, and the price of each view is determined through this bidding system. These auctions take place in milliseconds and are invisible to the human eye. Yet advertising using visuals and catch phrases, such as TVCF (TV commercial film) and posters, still plays a strong role. It is expected that communication will become more multifaceted by combining these two types of advertising methods or adopting different methodologies.

The figure analyzes the diffusion of tweets on the iPhones/Androids. Understanding information diffusion in the context of social media is extremely important for marketers today.

Qualitative analysis such as “ethnography” is now gaining attention

Qualitative analysis is a marketing approach that has been attracting attention among companies in recent years. This analysis is opposite to mass data analysis and includes “behavioral observation” and “ethnography.” Although big data provides information on attributes, and the behavior and purchase histories of a consumer, it does not perceive why the person got on a train and stopped by a store, and how much trouble was involved in deciding between item A and item B. It is also difficult for big data to discover potential needs that have not even been noticed by the consumer. Ethnography understands humans in a comprehensive manner based on empathy with the persons concerned, yet it still requires time, effort, and skill to follow users and observe their behavior closely all day long. Even so, we still need to be aware that companies find value in ethnography, which is useful in improving the usability of products and creating new product concepts.

Follow influential tweets on Twitter

One of my research projects in relation to big data is the Twitter tweets of those whose popularity makes them very influential (influencers). To be more specific, I collected tweets on iPhones and Androids sent out in Japanese for about a six-month period, and then selected the users who had a large number of re-tweets (candidate influencers). The research further analyzed network characteristics surrounding these same users and what percentage of them really had an influence on the tweets of other people. In 2013, my coauthors and I presented a series of the findings from our joint research at the annual conference of the Japanese Society for Artificial Intelligence (JSAI) held in Toyama and the research conference of the Japan Institute of Marketing Science (JIMS) held in Nagasaki. Based on these findings, we have so far found that among the users with a large number of re-tweets (candidate influencers), there exist a certain number of users whose tweets are influential enough to elicit tweets from their followers, but the extent of this tweet-inspired influence is limited.

Hope for people with a masterful command of big data

Both “recommendations” on online shopping sites and “cross-selling” (the action of recommending additional products or services based on the product or service purchased by a customer) are known as examples of using customer data. These are systems that customize products according to the traits of the customers. These types of automated systems will develop further for the purpose of achieving individualized marketing and therefore foster a high level of brand loyalty among customers.

However, if it is necessary to utilize big data for advertising and marketing, the first requirement is to identify what information should be extracted from the data and know how it is going to be put to use in the business. I think the forthcoming challenge associated with big data will be how to attract people who have excellent data analysis skills and a good sense of marketing. “Our company has a mountain of data, so if we dig it up, there may be some useful information” – If we utilize big data with this level of mindset, it may prove disappointing. I wonder if Japanese companies can provide an environment in which professional marketers can deal better with big data. Firstly, they should continue accumulating small findings within a particular domain, and make good use of such data in developing a detailed approach to the customers and promotion in order to systematically increase the sales and market share. I believe taking these steady steps will become more important.

Profile

Associate Professor, School of Commerce, Department of Commerce, Meiji University

Marketing Division and R&D Division, Hakuhodo Inc. Assistant Professor and Associate Professor, University of Tsukuba, 2003
Associate Professor, School of Commerce, 2008 Ph.D. in Economics, University of Tokyo

Research interest: Marketing Science, Consumer Behavior Research, Complex Systems Science