JSON Usage in Python and Pandas

Python dict to json or vice  versus

• How To Convert Python Dictionary To JSON? 

Panda DataFrame and Series to JSON:

• pandas.Series.to_json 
• pandas.DataFrame.to_json 

JSON to javascript in Django framework

• How to pass data to javascript in Django Framework ? 
• JavaScript loop through json array 

Create JSON object dynamically via JavaScript (Without concate strings) 

xy_kline_data = []
for (var key in kline_data) {
    xy = { 
          x: new Date(key),
          y: [ kline_data[key]["open"], 
              kline_data[key]["high"], 
              kline_data[key]["low"], 
              kline_data[key]["close"]
          ]
    }
    xy_kline_data.push(xy)
}

Using Redis As Cache for Database in Django

• Caching a View in Django
• django-redis-cache Advanced Configuration 
  
  

How to Cache Using Redis in Django Applications

here is my views.py bases on above link (also its code on github):

from django.shortcuts import render
from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework import status
from django.core.cache import cache
from django.conf import settings
from django.core.cache.backends.base import DEFAULT_TIMEOUT

import string, random

CACHE_TTL = getattr(settings, 'CACHE_TTL', DEFAULT_TIMEOUT)
from .models import Product

# Create your views here.

def id_generator(size=6, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for _ in range(size))
    
def price_generator(size=3, chars=string.digits):
    return ''.join(random.choice(chars) for _ in range(size))
            
            
def create_product(name, desc='', price=0):
    product = Product(name=name, description=desc, price=price)
    product.save()
 
@api_view(['GET'])
def view_books(request):
     
    products = Product.objects.all()
    if len(products) < 1000:
        i = 0
        while i < 1000 - len(products):
            create_product(id_generator(), desc=3*id_generator(), price=int(price_generator()))
            i += 1

    results = [product.to_json() for product in products]
    return Response(results, status=status.HTTP_201_CREATED)

@api_view(['GET'])
def view_cached_books(request):
    if 'product' in cache:
        # get results from cache 
        products = cache.get('product')
        return Response(products, status=status.HTTP_201_CREATED)
    else:   
        products = Product.objects.all()
        results = [product.to_json() for product in products]
        # store data in cache
        cache.set('product', results, timeout=CACHE_TTL)
        return Response(results, status=status.HTTP_201_CREATED)

Here is requirements.txt (Be aware djangorestframework version):

$ pip freeze
asgiref==3.2.10
Django==1.9
django-redis==4.12.1
djangorestframework==3.6.3
pytz==2020.1
redis==3.5.3
sqlparse==0.4.1

Django: Highlight Search Results

Solution refer to below links

• stackoverflow: how-to-highlight-searched-queries-in-result-page-of-django-template
  
• Custom template tags and filters
• Highlight matched search term
  

Python re.sub() case insensitive replace coding example: 

In [54]: re.sub('(%s)'%re.escape('hE'), r'<abc>\g<1></abc>', 'hello', flags=re.IGNORECASE)
Out[54]: '<abc>he</abc>llo'


In [56]: re.sub('(%s)'%re.escape('hE.'), r'<abc>\g<1></abc>', 'he.llo', flags=re.IGNORECASE)
Out[56]: '<abc>he.</abc>llo'

Or another version (Here we exclude URL replacement from HTML text): 

    pattern = re.compile('(^%s|[\s]%s|%s[\s])' % tuple([re.escape(word),]*3), re.IGNORECASE)
    highlighted = pattern.sub(r'<span class="highlight">\g<1></span>', highlighted)

Regex resource refer to:

• regexr.com
• regex cheatsheet
  
  

Nested list comprehension syntax in Python

Refer to 

• Understanding nested list comprehension syntax in Python
  

Example: 

>>> non_flat = [ [1,2,3], [4,5,6], [7,8] ]
>>> [y for x in non_flat for y in x]
[1, 2, 3, 4, 5, 6, 7, 8]

The right way to create database for Django - collation error

The right way to create a database

Refer to 

• django-python-collation-error/7083511#7083511
• fix-django-illegal-mix-of-collations
  

When creating a new database, remember to create with the right collate settings:

CREATE DATABASE foo CHARACTER SET utf8 COLLATE utf8_general_ci;

Change MySQL default character set to UTF-8 in my.cnf? 

Error Number: 1267

Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '='

Solutions refer to :

• Illegal mix of collations MySQL Error
• Change MySQL default character set to UTF-8 in my.cnf?

config on /etc/my.cnf

[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8