Handle escaped braces in f-strings

[stephenfin]

To use a curly brace in an f-string, you must escape it. For example:

>>> k = 1
>>> f'{{{k}'
'{1'

Saving this as a script and running the 'tokenize' module highlights something odd around the counting of tokens:

$ python -m tokenize test.py
0,0-0,0:            ENCODING       'utf-8'
1,0-1,1:            NAME           'k'
1,2-1,3:            OP             '='
1,4-1,5:            NUMBER         '1'
1,5-1,6:            NEWLINE        '\n'
2,0-2,2:            FSTRING_START  "f'"
2,2-2,3:            FSTRING_MIDDLE '{'     # <-- here...
2,4-2,5:            OP             '{'     # <-- and here
2,5-2,6:            NAME           'k'
2,6-2,7:            OP             '}'
2,7-2,8:            FSTRING_END    "'"
2,8-2,9:            NEWLINE        '\n'
3,0-3,0:            ENDMARKER      ''

The FSTRING_MIDDLE character we have is the escaped/post-parse single curly brace rather than the raw double curly brace, however, while our end index of this token accounts for the parsed form, the start index of the next token does not (put another way, it jumps from 3 -> 4). This triggers some existing, unrelated code that we need to bypass. Do just that.

Closes: #1948

[stephenfin]

As noted in #1948, this is mainly preventing this code for executing:

flake8/src/flake8/processor.py

Lines 207 to 218 in 2a811cc

	if previous_row:
	(start_row, start_column) = start
	if previous_row != start_row:
	row_index = previous_row - 1
	column_index = previous_column - 1
	previous_text = self.lines[row_index][column_index]
	if previous_text == "," or (
	previous_text not in "{[(" and text not in "}])"
	):
	text = f" {text}"
	elif previous_column != start_column:
	text = line[previous_column:start_column] + text

However, I have no idea what that code is intended for nor if I can remove it outright. Hopefully someone here can enlighten me so I can stick a comment in for future contributors/future me.

[asottile]

please rebase out unrelated formatting changes

[asottile]

please write a separate test demonstrating your problem instead of changing the meaning of this test

[asottile]

I meant an entirely separate test -- with a name describing what's being tested (specifically the handling of curly braces in fstrings) rather than overloading the existing test

[stephenfin]

Seems a bit overkill, no? This accomplishes the same thing without significant additional runtime cost, and it's arguably a better test owing or the fact it mixes strings and "code". From your comment on the issue:

nope! the method is meant to redact string contents and not code

I can straight up copy-paste 99% of the test, but it just seems a bit daft.

[asottile]

I've asked for it twice now