I have about a hundred thousand files, for each file I want to do the following:
Between the fifth and sixth character in the file with the following ascii code:
0x1f, a string is present. Now I want the application to open a specific file containing a list of all the substitutions, see later for the format of that substitutions file. If the substitution is not in the file, write the filename to stderr so I can manually fix that one later. Now between the 16th and 17th character with ascii code
0x1f the thing to substitute will appear again, however this time the field is not just the thing to substitute, but a string of html containing whatever shall be substituted from one to several times. I only want the first occurence in that field replaced.
The format of the substitutions file is fairly simple, each substitution is on its own line and they're separated by a space. They're ordered by the length of the thing to substitute.
CCCC 3 BCC 233 CCA 331 CCB 332 ACC 133 AA 11 AB 12 BA 21 BB 22 CC 33 A 1 B 2
Note that it's not guaranteed to be characters and numbers like above, that's just an example and the might contain UTF-8.
File: (0x1f characters are written as ^_ in the following example)
field1^_field2^_field3^_field4^_field5^_BB^_field7^_hai this field contains a newline^_some UTF-8オイ^_the next field is empty^_^_ another newline^_field14^_field15^_<b>BB</b>stuff BB^_the previous field contains something to replace^_^_^_more fields...
This file would get turned into
field1^_field2^_field3^_field4^_field5^_22^_field7^_hai this field contains a newline^_some UTF-8オイ^_the next field is empty^_^_ another newline^_field14^_field15^_<b>22</b>stuff BB^_the previous field contains something to replace^_^_^_more fields...
A little background
Some idiot decided that instead of creating separate columns in our database, he made a single column and separated the fields with the 0x1f character. He also decided that it's OK to duplicate the information I want to change in two different fields. I extracted the information from the database into a file pr. row containing just the column with the fields, as I suspect that is easier to work with, however if you can make a statement I can give to a SQLite database, that's fine too.