Unclassified interesting Tech Material - page 8

Unclassified interesting tech material - page 8

-x-x-


####### Saturday 06 May 2023 08:51:38 PM IST #######
27.
Hopefully the last post on Blogger:
Getting 
https://github.com/sabrinaluo/gatsby-theme-replica
to work.
- upgrade to latest node, gatsby and other plugin versions
- preferably use npm let's see; I don't really mind yarn tho ! as long as it works.

Saw this error for the first time.
https://stackoverflow.com/questions/69692842/error-message-error0308010cdigital-envelope-routinesunsupported
- the plugin worked with node --> 14.21.3
- will try to upgrade the packages.
-x-x-
- trying to do  a yarn upgrade --latest (just blindly getting the latest packages )
- above did not work for the example workspace..
- need to get example to work before importing into my blog project
- will have to use yarn for my project as well due to this back dependency

https://stackoverflow.com/questions/72488958/false-expression-non-string-value-passed-to-ts-resolvetypereferencedirective
- new new errors - dekhne ko mil rahe hain.. :D

- so for yarn 1.x to upgrade packages to latest -->
workspace add <full list of packages>

 - github slugger version 2.0.0
stopped supporting require based imports.
Well everyone should move out ideally
generally no way around it..
on why require needs to be phased out
https://stackoverflow.com/a/56068668
Quoting
Why? Because CommonJS isn't statically analyzable in many edge cases, and therefore "bails out" of optimizations like tree-shaking, and scope-hoisting. This mean's your JavaScript (the most expensive resource on your website to load), will have all sorts of dead/unused code in it.
- x-x-x-

gatsby 5.9.0 seems to not be happy with the project, got stuck at build schema for quiet some time.
- trying 5.7.0

https://github.com/gatsbyjs/gatsby/issues/23062#issuecomment-628063809
seems like 2 conflicting versions of graphql get installed if there is only partial version upgrade when installing with yarn add.

Ok this was my fault !
forgot to update peerDependencies, devDependencies in one of the workspaces 

####### Sunday 07 May 2023 07:16:49 PM IST #######

- fix the graphql error arising out of the graph queries written with the expectation of the old mdx node structure.
- by manually adding the old key for nodes and updating the structure, since the type inference does not work well for fields with non-consistent values. 
followed this:
--> some hints for why type inference does not work for non-uniform fields
https://github.com/gatsbyjs/gatsby/issues/22345
helpful but did not work without type modification
https://paulie.dev/posts/2022/09/mdx-2-breaking-changes-and-gatsby-plugin-mdx-v4-slug/
Also I like the theme: (this is the same guy who made the terminal theme)

Of-course restarting once again is also a very good strategy:
 gatsby clean && gatsby develop

####### Monday 08 May 2023 11:37:55 PM IST #######
when mdx is missing on fields in graphql:
either your reading of mdx failed, or the create node error-ed out for some reason.
https://stackoverflow.com/questions/62209671/gatsby-keeps-complaining-cannot-query-field-fields-on-type-markdownremark-wh

- the interactive editor also helped a lot in exploring the schema


https://stackoverflow.com/a/64802753
Webpack has stopped auto inserting polyfills
now needs to be done explicitly, give the freedom to avoid unnecessary code in browser bundle
for gatsby specifically:




####### Wednesday 10 May 2023 02:57:49 PM IST #######

- tailwind classes have also changed a little
text-blue --> text-blue-500
fixed that in the plugin as well


- working with a gatsby local theme / plugin:
https://github.com/gatsbyjs/gatsby/tree/master/examples/using-multiple-local-plugins
-  most libs have to be packed and then linked with the resulting .tgz
https://github.com/gatsbyjs/gatsby/tree/master/examples/using-multiple-local-plugins


-x-x-x-
Getting the error when using the theme with just npm install pointing to package.json

'Error: TypeError[File.publicURL]: Cannot convert to OutputType the following value: Object({ type: GraphQLScalarType
the error:
the solution for the error:
yarn link, npm link don't resolve dependencies correctly
https://github.com/gatsbyjs/gatsby/issues/20718#issuecomment-648284205
suggest connect-deps package:
https://github.com/gatsbyjs/gatsby/issues/29774#issuecomment-797305528
following the process manually in the readme here:
The plugin works.


-x-x-x-
####### Sunday 07 May 2023 07:51:46 PM IST #######
- most of the errors are due to changes in the gastsby-plugin-mdx library
- may be replace the plugin for something simpler 
- or need to find a way to get generated html from the plugin to the component for rendering.

####### Monday 08 May 2023 11:57:41 PM IST #######
https://www.gatsbyjs.com/docs/how-to/routing/mdx/#programmatically-creating-pages
- this guide was the most helpful when correcting compatibility issue with the plugin-mdx lib 

- had to decompose everything and start ground up to get a base working version of .mdx being correctly parsed and rendered.
- then slowly added components one by one, correcting relevant errors as new things were added
- credits to gatsby and react ecosystem, the errors were meaningful and pointed in the right direction mostly
- and lo and behold ! the theme has started to work in the example project 

Next:
- use it with an external gatsby project !
https://www.gatsbyjs.com/docs/creating-a-local-plugin/
- excellent guide


####### Saturday 06 May 2023 08:53:41 PM IST #######
26.
yarn workspace are great !


####### Saturday 06 May 2023 06:20:30 PM IST #######
28.
Npm execute a command in an installed npm module
https://stackoverflow.com/a/48803884


####### Thursday 27 April 2023 07:25:22 PM IST #######
29.
By far the best article / explanation of the python GIL I have come across 
https://realpython.com/python-gil/

####### Thursday 27 April 2023 03:40:17 PM IST #######
30.
My learning philosophy:

You can never understand everything. But, you should push yourself to understand the system.

- Ryan Dahl (Creator of Node JS)



+ have a general understanding of all parts of the system
+ openness to dive deep when required
+ do not wait for 100% system understanding to be productive
+ black-box understanding of systems too goes a long way in adding to the current part of the system you are working on.

I pride myself on being able to "see the forest for the trees and simultaneously be able to focus on a tree and fix it if required :D"
- this has got to be my LinkedIn status
- a silicon to server to space engineer - able to "see the forest for the trees and simultaneously be able to focus on a tree and fix it if required :D"

So, I like to believe I can see the forest for the trees, and at the same time can go get my toolkit and fix a tree if required.
####### Monday 03 July 2023 04:27:30 PM IST #######
Even,
S Jaishankar agrees that this is a rare quality:
https://youtu.be/yVTNge3sXpg?t=630


####### Monday 03 July 2023 08:28:26 PM IST #######
In my mind this alludes to the levels of executive and the on-ground worker as well

If you can talk with crowds and keep your virtue,
Or walk with Kings—nor lose the common touch,

- poem: If, Rudyard Kipling 

####### Wednesday 05 April 2023 09:03:47 PM IST #######
31.
Auto mounting ntfs partitions:

https://askubuntu.com/a/1235221
- edit the options using disks (ubuntu installed by default)
- mostly has sane defaults
- tweak the options if necessary

https://askubuntu.com/questions/46588/how-to-automount-ntfs-partitions
- other methods are useful as well
- quickest is cli - if you are fluent


https://askubuntu.com/questions/257066/mount-ntfs-partition-with-write-access-for-everyone
- was not able to find the "permissions" option in the latest documentation.. 
so did not try it as of now


####### Thursday 30 March 2023 11:51:23 AM IST #######
32.
Tunnel and use a remote machine as a proxy
ssh -D <port>


Hopefully this is the last post on blogger -> moving to a local static site generator + github pages 
33.
####### Sunday 19 March 2023 01:36:06 PM IST #######

- static site generator
- hugo
may be just stay in the javascript eco system for now

- gatsby
####### Saturday 06 May 2023 08:49:58 PM IST #######
https://github.com/sabrinaluo/gatsby-theme-replica
- very well made but old
- will need to upgrade, but the project structure follows the theme pattern well.
- can be just plugged into another project.

####### Sunday 07 May 2023 07:53:33 PM IST #######
-  loved the theme --> seems like a a version on the terminal theme !
nice !


- hexo
seems to have a lot of chinese influence - hard to understand the templates

- metalsmith
--> not many themes, less usage

- docusaurus
https://wiki.nikiv.dev/#getting-started

- mkdocs
a bit on the bland side



33.5
install oh-my-zsh 


34.
####### Sunday 19 March 2023 08:12:45 PM IST #######
install node and npm
- best way to manage these

35.
###### Sunday 19 March 2023 04:32:58 PM IST #######
fund raising:

36.
####### Sunday 19 March 2023 04:47:04 PM IST #######
knowledge graph

37.
####### Sunday 19 March 2023 06:19:24 PM IST ##3####
download a static site using wget
wget --limit-rate=200k --no-clobber --convert-links --random-wait -r -p -E -e robots=off -U mozilla http://www.example.com

- parsing the blogger xml and converting to various formats



38.
Good system monitor applet for gnome system:

base repo:
https://github.com/paradoxxxzero/gnome-shell-system-monitor-applet

the fork used to build the bleeding edge:
https://github.com/mgalgs/gnome-shell-system-monitor-applet

- great user experience once chrome gnome shell is installed correctly


a few more suggestions:
conky
https://askubuntu.com/a/1014284


####### Saturday 18 March 2023 02:23:42 PM IST #######
39.
a good list of vm (virtual machine) softwares available for linux:
https://www.fosslinux.com/48755/top-opensource-virtualization-software-for-linux.htm
- good argument for checking out gnome-box


other candidates:
- multipass 
- virt-manager



- boxes --> perfect for use if can be configured to take disk space on another drive and not crowd the home or root directories
-x-x-x-
this is done !
###### Saturday 18 March 2023 03:53:48 PM IST #######

currently the software does not allow this to be a setting.. duh !

Soooo,
Basically symlink the damn thing !!
https://www.techrepublic.com/article/change-drive-location-gnome/


###### Sunday 19 March 2023 08:16:34 PM IST #######
Ok, basic operation of cloning a repo does not work well here.
neither for boxes, nor for virt-manager
issue discussed in detail here:
https://stackoverflow.com/questions/65650299/gnome-boxes-on-fedora-33-fails-to-open
need to try mutipass now.



####### Friday 17 March 2023 02:58:53 PM IST #######
40.
Getting the transcript for a youtube short

https://www.reddit.com/r/youtube/comments/10m6ojo/how_do_you_pull_transcripts_from_youtube_shorts/

- basically the same short video can be opened up as a regular youtube video
and the transcript functionality works the same

####### Sunday 12 March 2023 02:50:06 PM IST #######
41.
manage docker as a non-root user
- very useful for running scripts without sudo

note
- needs a restart to take effect 

- If you initially ran Docker CLI commands using sudo before adding your user to the docker group, you may see the following error:

WARNING: Error loading config file: /home/user/.docker/config.json -
stat /home/user/.docker/config.json: permission denied


####### Sunday 12 March 2023 01:38:44 PM IST #######
42.
IoT data management

- written in java
- supports mqtt and http apis

##### Sunday 12 March 2023 01:06:35 PM IST #######
43.
git switch off checking filemode / file-permission changes 

git config core.filemode false

####### Saturday 04 March 2023 02:14:01 PM IST #######
44.
- citing expressjs example:
- the difference between and res.write and res.send 
- send writes and closes the connections 

####### Friday 03 March 2023 12:09:46 PM IST #######
45.
Using google sheets as db and backend 

####### Thursday 02 March 2023 11:48:10 PM IST #######
46.
Using youtube to store data
- project
https://github.com/DvorakDwarf/Infinite-Storage-Glitch

####### Sunday 26 February 2023 06:32:06 PM IST #######
47.
interacting with kernel modules - 

####### Sunday 26 February 2023 06:23:55 PM IST #######
48.
When you bluetooth is stuck on linux -
https://linuxhint.com/restart-bluetooth-ubuntu/

- reload the kernel module

###### Sunday 26 February 2023 05:30:49 PM IST #######
49.
How to incorporate rust into your stack
https://www.propelauth.com/post/i-love-building-a-startup-in-rust-i-wouldnt-pick-it-again

- speed up the slowest parts, only build what is absolutely necessary



####### Sunday 26 February 2023 04:57:02 PM IST #######
50.

####### Sunday 26 February 2023 04:00:35 PM IST #######
51.
- github search engine written in rust



###### Tuesday 14 February 2023 01:17:27 PM IST #######
52.
Get your google inbox offilne
https://www.howtogeek.com/709718/how-to-open-an-mbox-file-in-mozilla-thunderbird/

- great filters in thunderbird 
- can help you keep your inbox clean and save space on gmail.
- very fast - ~10GB mailbox can be searched through in a few seconds.


####### Sunday 12 February 2023 07:52:08 PM IST #######
53.
Change google drive folder permissions recursively:
- no other good method available in google drive

script:



####### Sunday 12 February 2023 01:48:31 PM IST #######
54.
React native - Duplicate Resources error
https://stackoverflow.com/a/57975287/2973457

- I am not sure why is the a in issue for such a long time
- cli script copying duplicate resources into the android folder.. don't know why 

rm -rf ./android/app/src/main/res/drawable-* rm -rf ./android/app/src/main/res/raw


####### Saturday 11 February 2023 12:12:01 AM IST #######
55.
Motion Canvas for technical animation - 
into:
https://www.youtube.com/watch?v=H5GETOP7ivs



-x-x-x-
great channel by the maintainer on the animation:
https://www.youtube.com/@aarthificial

This is similar to:
https://swlearninglog.blogspot.com/2022/01/unclassified-interesting-tech-material.html
point no 88.
about manim by 3blue1brown.

####### Thursday 09 February 2023 11:49:10 PM IST #######
56.
Some setups that I want to try out:
TODO: - checkout terminal environments that display ram, cpu, disk, and network usage at the bottom of the terminal.
TODO: - screen and audio recording tools that give options to start and stop recording on the screen itself
TODO: - desktop environment that display ram, cpu, dick and network usage at the top or bottom of the screen


####### Wednesday 08 February 2023 09:05:47 PM IST #######
57.
This is a cool project: a circuit that behaves like a guitar string
https://github.com/htfab/ks-guitar-2s
- a circuit to model a guitar string !!
- Strings can be set up, tuned, plucked or muted via the wishbone bus. The default build contains a single string, but it can be reconfigured for multiple strings with sympathetic resonsances by editing defines.v.


####### Wednesday 08 February 2023 03:02:13 PM IST #######
58.

- make the file name .zip
- and extract

####### Monday 06 February 2023 10:48:58 PM IST #######
59.
Error chain react native build - error linking resources:
https://github.com/negativetwelve/react-native-ux-cam/issues/105#issuecomment-1310104782

- also got to know the project was moved to a new repo :D :face_palm
and we were using some beta version

- was said to be fixed here:
https://github.com/facebook/react-native/issues/35210

####### Sunday 05 February 2023 07:21:24 PM IST #######
60.
Biggest misconceptions of open source 
https://betterprogramming.pub/the-biggest-lie-in-open-source-de38f71aa88c

####### Sunday 05 February 2023 05:58:15 PM IST #######
61.
Good overall explanation for deep linking:
https://medium.com/bumble-tech/universal-links-for-android-and-ios-1ddb1e70cab0

####### Sunday 05 February 2023 02:46:22 PM IST #######
62.
Linux observability tools by layer -

- nicely done bro !
- I don't have the source url

####### Sunday 05 February 2023 11:16:05 AM IST #######
63.
Phishing - 
http://globlhronickftccon.shop/?c=66
- shows a full screen page (so that a user thinks they are stuck)
- the page looks like a browser tab
- the looks a lot like the delhi police official page
- tries to get users to pay - accusing them of getting caught watching porn :D
- also plays the police siren sound 



####### Sunday 05 February 2023 12:01:49 AM IST #######
64.
A java and c# based tool for cloud file management
https://docs.cyberduck.io/cyberduck/
- good GUI and CLI options
- supports most major services


####### Saturday 04 February 2023 09:23:45 PM IST #######
65.
tar all sub-dirs inside a dir:

#!/bin/bash

echo "going to tar all dirs, in this path:" `pwd`

for i in *
  do
    tar -czvf "$i.tar.gz" "$i"


####### Saturday 04 February 2023 12:14:40 PM IST #######
66.
https://luplab.gitlab.io/rvcodecjs/

riscv instruction decoder
- It should be fun to build an assembly reader that can show instructions in plain english say in the right hand side panel !!
that is something nice to build really using this library
- step 2 - decode binary files --> and get them to show up like an assembly file - with human decoding on the right !!



##### Friday 03 February 2023 11:07:41 PM IST #######
67.
Tiny riscv emulator , without an MMU, memory management unit - very educational
https://www.youtube.com/watch?v=YT5vB3UqU_E    
- running linux in a pixel shader
- kernel debugging tips
- processor debugging tips
- make --> what is
- building linux from scratch 
- qemu - simulation
- bare-metal simulation - directly in assmebly 
other good option is to look at core dump / the way a JTAG can give you the system state

- executing code outside the emulator, from within the emulator NOT SAFE

https://luplab.gitlab.io/rvcodecjs/

instruction decoder

- exploring the linux loading mechanism 
- exploring the binary the kernel compiles into
https://blog.adafruit.com/2022/11/30/writing-a-really-tiny-risc-v-emulator-riscv-emulation-cnlohr/


####### Friday 03 February 2023 10:56:21 PM IST #######
68.
Booting Linux on a pixel shader - This guy is absolutely crazy 
- but why tho ?
- but you are awesome bro :D
- riscv arch

####### Tuesday 31 January 2023 08:24:23 PM IST #######
69.
Zephyr - RTOS 
- did not know that zephyr was such a popular RTOS 
https://docs.zephyrproject.org/latest/introduction/index.html

####### Thursday 26 January 2023 09:40:14 PM IST #######
70.
appimage - package once - run everywhere (linux)

https://appimage.org/


####### Thursday 26 January 2023 08:05:08 PM IST #######
71.
Adding app image based executable to the application menu    
https://codebysamgan.com/how-to-add-appimage-application-to-menu-in-ubuntu-linux

#### Thursday 26 January 2023 07:27:06 PM IST #######
72.
OpenSCAD - edit STL files / parts.
http://myrobotlab.org/content/using-openscad-edit-stl-files
- basically either modify it point by point 
- use it as a base part and add or subtract space / features to the part

72.5
OpenSCAD - print alphabets shaped extrusions example:
https://files.openscad.org/examples/Basics/text_on_cube.html 
- next thing I print will have the part number printed on it  ðŸ¤Ÿ


####### Tuesday 24 January 2023 10:48:21 PM IST #######
73.
require and module in the nodejs global scope and the steps they follow
https://www.freecodecamp.org/news/requiring-modules-in-node-js-everything-you-need-to-know-e7fbd119be8/

- amazing article
- gist:
  1. Resolving: To find the absolute path of the file.
  2. Loading: To determine the type of the file content.
  3. Wrapping: To give the file its private scope. This is what makes both the require and module objects local to every file we require.
  4. Evaluating: This is what the VM eventually does with the loaded code.
  5. Caching: So that when we require this file again, we don’t go over all the steps another time.

https://www.freecodecamp.org/news/requiring-modules-in-node-js-everything-you-need-to-know-e7fbd119be8#all-code-you-write-in-node-will-be-wrapped-in-functions

- this was what I was looking for when I stumbled across this article

####### Sunday 22 January 2023 04:33:02 PM IST #######
74.
Security breaches as on 6th Jan 2023:
- usually there's a cascading of breaches 
- since, data from one service can be instrumental in hacking allied or dependent services

####### Sunday 22 January 2023 04:29:09 PM IST #######
75.
good to haves that should become universal for programming languages:
https://buttondown.email/hillelwayne/archive/microfeatures-id-like-to-see-in-more-languages/
TODO: re-read

###### Sunday 22 January 2023 01:11:43 AM IST #######
76.
nodeJs - new object in loop - memory leak
- completion of the heapdump saga - from page 7 and 8
- the issue has been resolved.
- hence the dumps have been deleted.

it was a bug in node-js; 
where calling `new <Class>` in a for loop / repeated callback, leaks memory 
it is a known issue - claimed to be fixed, but well ... it happened :P

what led me to this issue:
this thread:
https://github.com/nodejs/help/issues/1484
and the comment:
https://github.com/nodejs/help/issues/1484#issuecomment-419422063

issue:
https://github.com/nodejs/node/issues/22229

pull request: (merged in 10.9.0)
https://github.com/nodejs/node/pull/22241

Other useful reading:
https://github.com/nodejs/node/issues/12805
-- insightful thread
-- similar symptoms
-- looked similar - but was not the cause for us

####### Friday 13 January 2023 01:49:01 PM IST #######
77. 
Kibana plugins for additional type of visualisations:
https://logz.io/blog/kibana-plugins-to-spice-up-data-visuals/
- sankey diagram
- swimlanes
- wafermap
https://www.outbrain.com/techblog/2017/01/kibana-for-funnel-analysis/
https://github.com/outbrain/ob-kb-funnel
- event funnel, user funnel
https://vega.github.io/vega/examples/
- vega to customize graphs - examples


####### Wednesday 11 January 2023 03:53:30 PM IST #######
78.
git diff exclude file / path

git diff develop -- . ':!package-lock.json'

tried all the combos only the above one worked.
for git version 2.25.1
from:


####### Sunday 08 January 2023 04:30:38 PM IST #######
79.
Apache mod_rewrite --> 
path info postfix and per-dir prefix bug 

where I first found out about it (after hours of banging my head):
https://stackoverflow.com/a/5520004/2973457

The bug thread:
https://bz.apache.org/bugzilla/show_bug.cgi?id=38642
banging my head be like:  https://bz.apache.org/bugzilla/show_bug.cgi?id=38642#c30
correctly captures my emotions

the documentation never clearly talks about what sets the PATH_INFO variable in the first place  



####### Sunday 08 January 2023 04:19:23 PM IST #######
80.
Apache logging and debugging tips and tricks
- global logging levels
- directory level logging levels
- module level logging levels
https://serverfault.com/a/158369

permutation and combination of all three can be used, with the more specific one overriding the more generic one.

- nice one apache team !

####### Wednesday 04 January 2023 04:40:01 PM IST #######
81.
Always test nginx config before restarting

sudo nginx -t

- do the same for apache as well

####### Monday 02 January 2023 09:25:22 AM IST #######
82.
wordpress plugin to serve the same wordpress site from multiple domains:
- it works well for simply replicating the site on another domain
https://github.com/straube/multiple-domain
https://wordpress.org/plugins/multiple-domain/

####### Wednesday 28 December 2022 12:20:54 AM IST #######
83.
setting https mode for proxied for wordpress 

-
official documentation:
https://wordpress.org/support/article/administration-over-ssl/#using-a-reverse-proxy
- never mentioned what it needs to be put before :facepalm
(although it may be general knowledge for wordpress developers, but koi to kahin to bata deta bhai !! )

Koi to bata deta bhai.. 2-3 days were lost on this:

You need to add the HTTPS/SSL settings before require_once(ABSPATH . 'wp-settings.php'); in your wp-config.php

So the relevant part of your wp-config.php should look like this.

define( 'FORCE_SSL_ADMIN', true );

// in some setups HTTP_X_FORWARDED_PROTO might contain
// a comma-separated list e.g. http,https
// so check for https existence
if( false !== strpos( $_SERVER['HTTP_X_FORWARDED_PROTO'], 'https' ) ) {
    $_SERVER['HTTPS'] = 'on';
}

/** Sets up WordPress vars and included files. */
require_once( ABSPATH . 'wp-settings.php' );




####### Tuesday 27 December 2022 11:19:34 PM IST #######
84.
Use Express server to serve some urls from angular universal and some from a proxy for the same root url, when there is no clear pattern to distinguish between the two servers



gist for express server to serve some registered urls from self and others from proxy
const express = require('express');
const httpProxy = require('http-proxy');
const app = express();
const proxy = httpProxy.createProxyServer();
const port = 3001;
const serverBehindProxy = 'https://a.b.c';
// const serverBehindProxy = 'http://localhost:3000';
const pathsAvailableOnWebsite = {
"":true, // root path (http://localhost:3001/base-url-for-proxy; in this case) to be served from the website and not from the proxy
"path-available-on-website-1" :true, // to be served from website angular
"path-available-on-website-2" :true, // to be served from website angular
}
app.get('/', (req, res) => {
res.send('Hello World! From proxy server root');
});
// add middleware for interception
app.use('/base-url-for-proxy*', (req, res, next) => {
// uncomment for debugging
// console.log("middleware invoked");
// console.log('Request Type:', req.method);
// console.log('requested path:', req.path);
// console.log('requested baseUrl:', req.baseUrl); // url without params and stripped of the mount path
// console.log('requested originalUrl:', req.originalUrl);
// if path not reserved to be served from the website;
// proxy to wordpress server
const subDirAfterOCC = req.baseUrl.replace("/base-url-for-proxy", "").replace("/", ""); // cascaded to handle root path
if (!pathsAvailableOnWebsite[subDirAfterOCC]) {
// console.log("path not available on website, try to server from proxy");
// console.log(`trying target: ${serverBehindProxy}${req.originalUrl}`);
proxy.web(req, res, {
target: `${serverBehindProxy}${req.originalUrl}`,
// secure: false,
// toProxy: true, // what stupidity is this ??? the options do not work as documented
// prependPath: true,
ignorePath: true, // this needs to be true if express is setting the req.url to / for file req making them look like directory req
changeOrigin: true,
xfwd: true
}, (err) => {
console.log("error proxy serving the request :(", err);
});
} else {
console.log("path available on website, let angular serve it, do nothing");
next();
}
});
// app.get('/base-url-for-proxy*', (req, res) => {
// res.send('Hello World! From proxy server base path');
// });
app.get('*', (req, res) => {
res.send('Hello World! From catch-all path');
});
app.listen(port, () => {
console.log(`Example app listening on port ${port}`);
});
const express = require('express');
const app = express();
const port = 3000;
app.get('/', (req, res) => {
console.log("original url", req.originalUrl);
res.send('Hello World! From the base express server behind the proxy');
});
app.get('*', (req, res) => {
console.log("original url", req.originalUrl);
res.send('Hello World! From the catch-all path of base express server behind the proxy');
console.log("req headers:", req.headers);
});
app.listen(port, () => {
console.log(`Example app listening on port ${port}`);
});
packages used:
https://www.npmjs.com/package/http-proxy#node-http-proxy
-- good package, lot of flexibility available
-- streams everything, so great scalability and low overhead
cons:
-- like everything else in the node eco-system has poor documentation, better read the code than the documentation. The options documentation really lacks clarity about what they affect.


####### Monday 12 December 2022 12:46:20 PM IST #######
85.
How to clear nginx cache:
https://www.sorendam.com/clear-nginx-cache/
- sometimes the server caches requests even without the caching headers in place (very weird)
- delete the contents of the folder.. not the folder itself, if the folder is deleted, nginx will refuse to start..


####### Thursday 08 December 2022 11:40:22 PM IST #######
86.
A Go-based static site generator:
https://gohugo.io/getting-started/usage/
- good for blogs, docs, etc
- pages / navigation is picked up from file system navigation 
- template + data in the folder --- hugo ---> rendered html page 


####### Thursday 08 December 2022 11:36:28 PM IST #######
87.
Tips to make faster landing pages:
https://unbounce.com/landing-pages/increase-landing-page-speed/
- chrome lighthouse is the goto tool of course for most of the devs / marketeers 

Landing page design tips:

####### Thursday 08 December 2022 08:42:36 PM IST #######
88.
The search for a fast low size framework / library for building responsive landing pages:

- gatsby
https://www.gatsbyjs.com/docs/quick-start/
- officially suggested by react for single page websites
https://reactjs.org/docs/create-a-new-react-app.html#recommended-toolchains
- fairly small bundle size (gzip, 30kb for hello world basic page)
- the starters did not get installed, but hello-world worked well
- simple routing: https://www.gatsbyjs.com/docs/how-to/routing/
- all pages are individually generated - (still need to test this), but that seems to be the case
- deploying gatsby on own server was a bit of a rare find:
https://dev.to/jdelvx/serving-a-gatsby-site-from-digitalocean-using-nginx-ahn
but the deployment is simple
- gatsby starters are not working for me at all:
https://www.gatsbyjs.com/starters/r-ichard/gatsby-starter-bootstrap-5

- react-bootstrap
https://react-bootstrap.github.io/getting-started/introduction/
- may try to add this on top of gatsby if required.
- but mostly planning to use only the grid from bootstrap.
- will be an additional 40kb download after gzip


- default react 
https://reactjs.org/docs/create-a-new-react-app.html
- for single page apps
- so not trying out 
some additions
####### Friday 06 January 2023 12:36:19 PM IST #######
- some advice to the keep the size down
https://rajaraodv.medium.com/two-quick-ways-to-reduce-react-apps-size-in-production-82226605771a#:~:text=If%20you%20are%20building%20a,redux%2Dblog%20(live).
- result - 90KB - still larger than gatsby 
- so still not trying

- react as a library
https://reactjs.org/docs/add-react-to-a-website.html
- will need toolchain setup
- may be if we can find a good boilerplate then this can be considered
- this may help with react as a library 
- worth a try
https://neutrinojs.org/packages/react/ 
- this configured for a single page app
- still looking for one tailored for - independent landing pages
boilerplate webpack for react + bootstrap 5
-
still looking a solution here
prospective candidates:
https://staticsiteboilerplate.com/
https://github.com/ericalli/static-site-boilerplate
https://github.com/WeAreAthlon/frontend-webpack-boilerplate
https://github.com/erickzhao/static-html-webpack-boilerplate



- preact
 
https://preactjs.com/guide/v8/differences-to-react
- did not want to spend time on something not well adopted, to be stuck without plugins


- craco 


- angular - with reduced bundle size
https://medium.com/angular-in-depth/optimize-angular-bundle-size-in-4-steps-4a3b3737bf45
- but hello world itself on angular is 230KB 
- plus then there is a lot of unused js that needs to load for angular core to work
- routing from angular also adds about 700-900 milliseconds of delay

this is some sound advice as well:
https://danielk.tech/home/angular-how-to-improve-bundle-size


- webpack bundle analyzer is a great tool for bundle composition analysis
https://medium.com/angular-in-depth/optimize-angular-bundle-size-in-4-steps-4a3b3737bf45#1134


- vuejs
https://bundlephobia.com/package/vue@3.2.45
- would the next candidate to try
- but gatsby beat it on size - so will try gatsby first
optimization advice for vue apps:
https://blog.42mate.com/vue-js-bundle-size-and-performance-optimizations-%F0%9F%8F%8E%EF%B8%8F/

 bootstrap 5 
https://getbootstrap.com/docs/5.2/customize/optimize/#lean-sass-imports
- planning to use the import only what you use from bootstrap 5 to keep the pages lean on css and js as well
- even if we end up using bootstrap



####### Sunday 04 December 2022 09:44:25 PM IST #######
89.
Sample data for ArangoDB:
https://github.com/arangodb/example-datasets
- comes handy for coding interviews / experimenting with queries etc




####### Sunday 04 December 2022 09:21:07 PM IST ######    
90.
Adding documentation to javascript and typescript code:

JSDoc
https://jsdoc.app/about-getting-started.html
- is a promising candidate 
- generates an HTML as per its template


API docs:
https://api-docs.io/
- free
- works with all swagger versions




####### Sunday 04 December 2022 09:12:21 PM IST #######
91.
Scheduled Tasks with nodejs:

-
Node Cron:
https://www.npmjs.com/package/node-cron
great lib !
- simple syntax 
- simple implementation
- using setTimeout repeatedly where job period is longer than max-interval for setTimeout - repeatedly set the max till the job timing is less than max milliseconds away..
issues:
-
graceful reload is an issue

may be bree can be tried:
BreeJS:
https://github.com/breejs/bree
 


####### Sunday 04 December 2022 09:04:30 PM IST #######
92.
Delay in FCM notifications:

- it is inevitable when devices are idle and heartbeat interval is longer

- of-course they are written by a competitor, but still the truth is the truth
- managing the fine balance between battery consumption and real-time notifications is very hard



93.
####### Sunday 04 December 2022 05:54:17 PM IST #######
Handling schema changes in elastic search / open search indices:
Update / Change kibana / elastic search / open search index pattern:

Elastic Dump - useful tool for managing kibana / elastic / open search indices 
https://github.com/elasticsearch-dump/elasticsearch-dump
- useful tool for backups, administration, 
password encoding is not very clearly mentioned in the documentation:
- only encode the password - not the entire url
https://stackoverflow.com/a/67743849/2973457
more about elastic dump:
https://stackabuse.com/guide-to-elasticdump-moving-and-saving-elasticsearch-indices/

Elastic / Open Search - If your "snapshot" utility is correctly configured, it is recommended over other methods
- very large backups can also be taken seamlessly




####### Sunday 04 December 2022 05:29:00 PM IST #######
94.



####### Saturday 03 December 2022 11:14:55 PM IST #######
95.
It is very cool of pm2 to not make this a paid feature:
pm2 monit    
looks very pretty on a terminal



####### Saturday 03 December 2022 11:11:22 PM IST #######
96.
Flame graphs for performance visualisation:
https://www.alxolr.com/articles/squeeze-node-js-performance-with-flame-graphs

- great to visually identify performance bottlenecks 
 


####### Saturday 03 December 2022 10:56:28 PM IST #######
97.
Streaming efficiently on nodejs:
https://nodejs.org/en/docs/guides/backpressuring-in-streams/
- this is a good read
- general thumb rule
- listen for buffer full of writer / sink
- stop reading / sourcing till the writer buffer is cleared
- listen for write complete / buffer empty
- start sourcing / reading again 
repeat till the task is done :)

An example write heapdump with a 1MB buffer
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
// heap dump using v8 api in the main process
function _createHeapSnapshot(filePrefix="node") {
  console.log(`Generating the heap snap shot with prefix =>${filePrefix}`);
  
  dumpingHeap = true;

  const snapshotStream = v8.getHeapSnapshot();
  // It's important that the filename end with `.heapsnapshot`,
  // otherwise Chrome DevTools won't open it.
  const fileName = `${filePrefix}-oms-jobs-${Date.now()}.heapsnapshot`;
  const heapFileStream = fs.createWriteStream(fileName, {
    highWaterMark: 1000 * 1024
  });

  heapFileStream.lastLogWrittenAt = 0;

  // console.log(`Trying to generate the heap snapshot using getHeapSnapshot`);

  snapshotStream.on("data", (chunk) => {
    // console.log(`Received ${chunk.length} bytes of data.`);

    let writeBufferEmpty = heapFileStream.write(chunk);

    if (!writeBufferEmpty) {
      // pause the read stream
      // console.log("Write buffer full, pausing reading");
      snapshotStream.pause(); 

      // console.log(`Write buffer full, will wait for drain to complete`);

      let bytesWrittenInMB = Math.round(heapFileStream.bytesWritten / bytesInMB);
      console.log(`Bytes written in MB ${bytesWrittenInMB}`);

      if (bytesWrittenInMB - heapFileStream.lastLogWrittenAt > 10) {
        console.log(`${bytesWrittenInMB}MB written to file`);

        heapFileStream.lastLogWrittenAt = bytesWrittenInMB;
      }
    }
  });

  heapFileStream.on("drain", () => {
    console.log("Writer drain complete, resuming reading");
    // resume read stream
    setTimeout(() => {
      snapshotStream.resume();
    }, 1000);
  });

  snapshotStream.on('end', () => {
    console.log("snapshot read complete, freeing resources");
    snapshotStream.removeAllListeners();
    snapshotStream.destroy();

    heapFileStream.close();
    dumpingHeap = false;
  });

  // snapshotStream.on("close", () => {
  //   console.log("heap dump generation complete ");
  //   dumpingHeap = false;
  // });
  
  return [snapshotStream, heapFileStream];
}

almost works as efficiently as the package written in C++
https://github.com/bnoordhuis/node-heapdump



####### Saturday 03 December 2022 10:44:30 PM IST #######
98.
Adding and enabling swap on linux systems

general intro to swap space:
https://opensource.com/article/18/9/swap-space-linux-systems
add swap:
https://www.cyberciti.biz/faq/linux-add-a-swap-file-howto/
move swap from one location to another:
https://www.cyberciti.biz/faq/ommand-to-move-swap-space/

- especially useful when it is known that some operation that needs to be performed will exceed the total ram on a system
- swap to be on in general for a ssd hard-disk is not so good for it's life.
https://askubuntu.com/questions/652337/why-are-swap-partitions-discouraged-on-ssd-drives-are-they-harmful



####### Saturday 03 December 2022 02:45:16 PM IST #######
99.
Angular JS and performance


- simple hello world html page:


- now trying out angular hello-world









####### Tuesday 29 November 2022 01:43:15 PM IST #######
100.
Getting the best possible image size and quality compromise


- The monitor I use has a 82 PPI density and 1920x1080 resolution, so a decent machine overall
- will try to chart out the image sizes at this resolution

This is a great article:
https://www.foregroundweb.com/image-size/

####### Saturday 03 December 2022 10:13:25 PM IST #######
Some thumb rules:
-


- the JPEG can be replaced with webp now.. better compression in general
webp vs jpeg comparison:
https://stackoverflow.com/questions/52246067/jpg-vs-compressed-jpg-vs-webp-why-webp-isnt-the-smallest-one


100.1
- this is also a good check list for web-page optimization:
https://web.dev/fast/











Continued from:
https://swlearninglog.blogspot.com/2022/05/unclassified-interesting-tech-material.html






Comments

Popular Posts