Analytics, Bigdata, Hadoop, Python

How to convert Python list, tuples, strings to each other…

There are three built-in functions in Python : lists, tuples, and strings. The three functions, str (), tuple (), and list (), convert to each other using the following example: >>> s = ‘123456’ >>> list(s) [‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’] >>> tuple(s) (‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’) >>> tuple(list(s)) (‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’) >>> list(tuple(s)) [‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’] >>> “”.join(tuple(s)) ‘123456’ >>> “”.join(list(s)) ‘123456’ >>> str(tuple(s)) “(‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’)” >>> str(list(s)) “[‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’]”

Analytics, Apache Spark, Hadoop, Kafka, Python, Spark

Consume JSON Messages From Kafka Using Kafka-Python’s Deserializer

Hope you are here when you want to take a ride on Python and Apache Kafka. Kafka-Python is most popular python library for Python. For documentation on this library visit to page https://kafka-python.readthedocs.io/en/master/. kafka-python is designed to function much like the official java client. kafka-python is best used with newer brokers (0.9+), but is backwards-compatible with older versions (to 0.8.0). Some features will only be enabled on newer brokers. So instead of showing you a simple example to run Kafka Producer and Consumer separately, I’ll show the JSON serializer and deserializer. Preparing the Environment Lets start with Install python package using […]

Python

pyshark, tshark and wireshark installation…

Python wrapper for tshark, allowing python packet parsing using wireshark dissectors. Installation All Platforms We are going to use python pip for installation if you dont have pip please follow below command to install pip:- # sudo yum install python-pip # sudo yum install python-wheel Once done  install pyshark using pip:- #pip install pyshark Now install tshark as pip does not identify it we go with yum whatprovides tool:- # yum whatprovides *tshark* confirm tshark version once done:- #tshark -v Now install wireshark #yum install wireshark Now go to python shell and use below command to sniff into network:- >>import pyshark >>capture […]

Python

Content Data Store(CDS) Compressing and enhancing technique…

Aggressively we are adding new features to Content Data Store(CDS) system. One of the feature that i am going to discuss here is compression technique(BigData application is incomplete without compression). And what if i tell you in CDS, we use compression along with enhancement of visual image/scanned documents. Our compression technique has two additional features:- Smaller:- Reduce file size and save 80% space compare to your image/scanned document. Clearer:- Isolate foreground color by identifying background color and choose small number of representative colors. Another important feature is performance. We don’t use api provided by office-lens or others, instead we have small python […]

Framework, Python

Almost Everything in Python!!!

A curated list of Python frameworks, libraries, software and resources. Inspired by awesome-php. Awesome Python Environment Management Package Management Package Repositories Distribution Build Tools Interactive Interpreter Files Date and Time Text Processing Specific Formats Processing Natural Language Processing Documentation Configuration Command-line Tools Downloader Imagery OCR Audio Video Geolocation HTTP Database Database Drivers ORM Web Frameworks Serverless Frameworks Permissions CMS E-commerce RESTful API Serialization Authentication Template Engine Queue Search News Feed Asset Management Caching Email Internationalization URL Manipulation HTML Manipulation Web Crawling Web Content Extracting Forms Data Validation Anti-spam Tagging Admin Panels Static Site Generator Processes Concurrency and Parallelism Networking WebSocket […]

Hadoop, Hive, Java, Pig, Python

Python and Python bites

Python and Python bites “lambda”    Hi everyone, this article show you one powerful function in Python programming language called “lambda”. It can solve any small problem in single line code. So lets start the beginning of your interesting or may be future programming language. Anonymous functions created at runtime are known as lambda functions. The below line defines an ordinary function usage in python. >>def f (x): return x+42 >>print f(21) 63 For lambda functions, >>calc = lambda x: x+42 >>calc(21) 63   lambda definition does not include a “return” statement. It always contains an expression which is returned. Also […]