Normalize special font characters #3

New Issue

kj7rrv · 2022-03-05T07:59:48-08:00

kj7rrv commented

2022-03-05 07:59:48 -08:00

Some spam messages on the Pine64 chat look like this:

𝐈 𝐬𝐭𝐚𝐫𝐭𝐞𝐝 𝐰𝐢𝐭𝐡 $1000 𝐚𝐧𝐝 𝐚𝐟𝐭𝐞𝐫 𝐬𝐞𝐯𝐞𝐧 𝐝𝐚𝐲𝐬 𝐎𝐟 𝐭𝐫𝐚𝐝𝐞𝐬 𝐈 𝐫𝐞𝐜𝐞𝐢𝐯𝐞𝐝 $10,500 𝐚𝐥𝐥 𝐭𝐡𝐚𝐧𝐤𝐬 𝐭𝐨 @Brandonholden 𝐟𝐨𝐫 𝐦𝐚𝐤𝐢𝐧𝐠 𝐢𝐭 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐦𝐚𝐲 𝐆𝐨𝐝 𝐛𝐥𝐞𝐬𝐬 𝐲𝐨

CedarSentinel doesn't always detect them. GPTC should probably convert them to regular text:

I started with $1000 and after seven days Of trades I received $10,500 all thanks to @Brandonholden for making it possible may God bless yo

This can be done with unicodedata.normalize("NFKD", text).

Some spam messages on the Pine64 chat look like this: 𝐈 𝐬𝐭𝐚𝐫𝐭𝐞𝐝 𝐰𝐢𝐭𝐡 $1000 𝐚𝐧𝐝 𝐚𝐟𝐭𝐞𝐫 𝐬𝐞𝐯𝐞𝐧 𝐝𝐚𝐲𝐬 𝐎𝐟 𝐭𝐫𝐚𝐝𝐞𝐬 𝐈 𝐫𝐞𝐜𝐞𝐢𝐯𝐞𝐝 $10,500 𝐚𝐥𝐥 𝐭𝐡𝐚𝐧𝐤𝐬 𝐭𝐨 @Brandonholden 𝐟𝐨𝐫 𝐦𝐚𝐤𝐢𝐧𝐠 𝐢𝐭 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞 𝐦𝐚𝐲 𝐆𝐨𝐝 𝐛𝐥𝐞𝐬𝐬 𝐲𝐨 CedarSentinel doesn't always detect them. GPTC should probably convert them to regular text: I started with $1000 and after seven days Of trades I received $10,500 all thanks to @Brandonholden for making it possible may God bless yo This can be done with `unicodedata.normalize("NFKD", text)`.

kj7rrv added the

enhancement

label 2022-11-25 10:16:12 -08:00

kj7rrv added the

model-break

label 2022-11-27 09:38:30 -08:00

kj7rrv added the

wait-for-break

label 2022-11-27 10:44:48 -08:00

kj7rrv referenced this issue from a commit

2022-12-24 11:18:41 -08:00

Normalize characters