What is Big data

Shopping habits reveal even the most personal information.  All major retailer vendors from investment banks, to sim card vendors, to grocery chains, bankers today is moving with a “predictive analytics” department specialized to comprehend the consumers’ shopping habits including their personal habits, to customize their marketing methods. For example Target a retail unit was able to predict based on the buying pattern,  that a  teenage shopper was pregnant before her dad did. The high school student’s father was furious as he stormed  into a Minneapolis-area store clutching coupons sent to his daughter for maternity and baby products. Such is the power of Data analytics.

Such data provide enough digital footprints for any interested party to do a follow-up. Going by the modern management theory of providing the customer with what he wants,  it is essential to understand what he wants. Though the data was being collected all along, Information technology was unable to handle provide an edge to business in terms of data analytics.

Is it only marketing.The answer is a definite NO. There are other areas of overlap which is attracting the attention of Management. Some of the other areas include, Security and Information Event Management which is a huge area of concern. The SIEM has started throwing digital foot prints on the application usage pattern of potential customers in terms of handling operating system and other applications which site on top of OS. Technologies like  SNMP  and RMON are capable of pushing specialized data to the external world which provides excellent insight.

Author-It for Management,Oxford Press

Logs - Sources of Big data

A log as understood is a record of the events occurring within a given organization’s systems and networks when viewed from a system perspective. They are composed of log entries where, each entry is composed of information pertaining to a specific event that has occurred within a system or network.  Having matured from a journal today logs contain information related to different types of events occurring within networks and systems. The data present in the log files describe the status of each component and record system operational changes, such as the starting and stopping of services, detection of network applications, software configuration modifications, and software execution errors. Logs are very much part of computing system and cannot be avoided. It is a common understanding that the data generated in the form of log by the existing network devices are a repository of information about the status of the network. Originally, the system of logs were used for troubleshooting problems. However logs now serve many functions within most organizations, such as optimizing system and network performance, recording the actions of users, and providing data useful for investigating malicious activity. 

Within an organization, logs contain records related to computer security; common examples of these computer security logs are audit logs that track user authentication attempts and security device logs that record possible attacks. Logs serve the purpose of journal or a day book and hold a record of all transactions that takes place in a network and provide a wellspring of information to help improve security, thus enabling  compliance. 

Log files are maintained in almost every system and they are usually examined during security audits, either external or internal. Indeed, during regular security audits, log files may be examined and correlated, in order to assure that the intended technical measures are in place and that the security policies and procedures are implemented. During non-scheduled security audits, e.g. as a response to a security incident, log files are analyzed in order to discover the cause of the incident, such as lack of security measures, non-conformance with security procedures or system miss configurations. Such logs form the basis of SIEM vertical. They serve as the source of analysis for a system study with the help of Big data. In other words SIEM is adopting itself to Bigdata. 

 

Author-IT for management, Oxford Press.