Skip sending oversized data to ES

[MinhNguyenDev]

I had a same issue as #112, one line in the log file was too big and when the ElasticsearchLogShipper tried to send the data, it failed and kept retry over and over again, which could be costly if using AWS Elasticsearch. Could we make a change to skip the large size record ? It will fail anyway and make the whole log shipper stop working and no way to recover unless we delete the line manually.
Here is some sample code:

                        while (count < _batchPostingLimit && TryReadLine(current, ref nextLineBeginsAtOffset, out nextLine))

                        {
                            if (nextLine.Length > 1000000) // We can make this number configurable
                            {
                                SelfLog.WriteLine("Line is too long. Length {0}. Skip sending to ElasticSearch", nextLine.Length);
                                continue;
                            }
                            // Add payload
                            ++count;
                        }

Thanks,

[mivano]

Sounds good. Care for a PR? Make it configurable would be good with a default that does not check for this?

[mivano]

Merged, closing issue