Really nice suggestions to improve your writing skills

Really nice suggestions to improve your writing skills

I really like and appreciate when messages are clear with CTA, easy to read and understand. This article contains ideas-to-follow with clear suggestions to follow with idea to improve:

https://www.app.college.police.uk/app-content/intelligence-management/analysis/delivering-effective-analysis/

Highly recommend.

Enjoy!

Storing secrets in git repository

Sometime you may want to store secrets in git repository and keep track of changes history. For example you may want to store a production configuration or any other sensitive information.

You may already know that git-crypt may be your friend in this case. But what you can do in case if you developing software using Windows platform like I'm doing this? That's may be a problem to use this nice tool on Windows OS.

You may find binaries for git-crypt somewhere on the internet, but would you trust to such binaries? I believe not.

What can we do in this case? We can build our own binary and test it on our repository.

In this article we will build such binary and will do few experiments to check that it will work for us.

Python: improving time required to load 1.5G CSV file

In previous post we discussed how we can search for specific item in 1.5G CSV file with 132M records. And we improved search from 73625.309 ms (more than a minute) to just ~0.005 ms - almost 15 million times faster. Which is pretty impressive improvement as per my understanding.

But there is still one bottle neck that can be improved - time required for first initial scan of the file. Let's try to improve this in this article.

All source files can be found in JFF-Bohdan/item_lookup

Python: playing with big lists (132M records), checking if item in list

Let's imagine situation when we need to check if item is in list and our list is pretty big. For example, we may have file with hundreds of millions records and we need to develop solution which should be able quickly say if we have specific item in that list or not.

Let's start with naive implementation and then try to improve it iteratively.

This post was inspired by post https://habr.com/ru/post/538358/

Охотничий кот

 Стандарты безопасности 60-х.

Итак, мой рассказ о дресированном охотничьем коте.

Simple package for duplicate files removal

Just created simple Python package that will use for duplicate images removal. Also may be used for identical files removal with any type of content.

Fast autocomplete using Python and Redis

Fast autocomplete using Python and Redis

General information

Problem

Imagine situation when you need provide autocomplete for some data list. In case when you using SQL probably you will do something like this:

select * from table_name where table_field like 'search%'

Probably you know, that when you database contains significant amount of data you will get essential decreasing of data processing.

N-Queens problem solving program

N-Queens problem solving program

N-Queen problem - you need N chess queens on a chess board with dimensions NxN in a way that they do not threat each other (according to chess rules).

Program can find all possible solutions for N-dimensional board. Also, program able to print available solutions to console or to the file.

Here is my solution in Python language: https://github.com/JFF-Bohdan/nqueens (MIT license)

Yet Another Tiny Backup

Yet Another Tiny Backup

General information

Backup tool for one of my production instances. It implements very simple logic for my special requirements.

Algorithm: yatbackup can compress whole folder using specified archiver. In case, when another backup with same hash available in destination folder, new archive will be deleted.

Yes, it's all.

Why do I need it? I need create backup of some folder using cron, but I don't need duplicate archives.

Also, I need to understand that utility successfully finished, even no new archives was created.

Source code and documentation available here: https://github.com/JFF-Bohdan/yatbackup

pynvr - Python NVR (Network Video Recorder) based on OpenCV.

pynvr - Python NVR (Network Video Recorder) based on OpenCV.

Source can be found here: https://github.com/JFF-Bohdan/pynvr

General information

pynvr can interract with cameras using OpenCV (supports many types of connection including connection to USB cameras and network cameras using RTSP, for example)

Captured video stream must be analysed for motion and then saved to files according to specific rules.

When motion detected (or rule triggered) not only video after start must be saved but some frames before (pre-alarm/pre-event frames) must be saved to output file.